yusuketsutsumi

Coding

My focus is on implementing technologies for back-end preformance and test evaluation.

Videos

Python Pet Peeves

01/26/2012 07:14pm | 0 Comments

As of this posting, Python has been my main programming language for over three years. Although I definitely feel that Python is not a good fit for all programming projects, the speed and efficiency with which I can code in it has made it my go-to language whenever possible. As such, I've seen a lot of Python code, and have had ample time to think about some of the more nuanced issues regarding coding standards. Here's a few of my pet peeves, and opinions about them:

from module import *

When I first started python, I used this particular import for a lot of things. I'm using so many methods from this module, why not just import the whole thing? It was definitely a pain in the neck fixing those include issues. Well, time in the industry has made me realize the error of my ways. This isn't just python related, this is related to any programming language. Includes/Import should always be as obvious as possible. The correct import methodology, is to do as such:

from module import w,x,y,z

or, if you want to be even nicer:

import module
module.x()

But what if we're using ten methods from that module? still gotta do it.
What about 20 methods? still gotta do it
What about 100 methods? don't know how there's 100 methods in a single module, but you still gotta do it.
The reasoning is simple: you're providing a very helpful hint that future coders can use to debug your code years from now. That hint is : where the method is actually found. While you yourself don't save any time off of doing this, you're saving hours of development time for future coders, giving them a roadmap to exactly what your function's stack actually is. Although this can be given by any IDE that has an understanding of the language and it's dependencies, one shouldn't assume that this is so. In my experience, when debugging, I have spent anywhere between a good ten to twenty minutes looking for methods, especially in python files with twenty lines of imports. To know exactly where a particular method or module comes from goes a long way to making one's code maintainable. For example, suppose I was a programmer who had to debug, and was able to pinpoint the bug to a method that had been previously written, called a_func. The file calling it looks like:

from foo import *
from bar import *

def b_func():
    ...
    a_func()
    ...
    return

Now if I had no knowledge of the modules foo and bar, I would have to look through BOTH foo and bar, and see if either of those had the function a_func. This is only a minor inconvenience if your code only has two of these imports, but the larger a script gets, and the more includes it brings in over the years, could result in one having to look through several files in various locations, to debug one call. Precious time that could have been saved, had the original code just written:

from bar import a_func

Use ternary's, but only where it makes sense

If you're not familiar with tenary operators, I'd suggest acquainting yourself now. After all, ternary operators only exist because the problem they solve is so prevalent in coding everywhere. Specifically, the strict point where you want a variable to be one of two things. In Python, ternary operators are represented differently than other programming languages (the typical ( condition ? do_this_if_true : do_this_if_false ) operation). Python has:

do_this_if_true if condition else do_this_if_false

Ternary's in general have several uses. The big one is providing a default value:

var = (value if value else default_value)

Basically, in any situation where you have:

if this:
	just_one_procedure()
else:
	just_one_other_procedure()

One should consider using a ternary. You can also nested ternarys, although I wouldn't suggest doing so for more than one level deep. This is especially useful when you have a variable assignment with four different possible outcomes:

x = ( (1 if a else 0) if b
else (2 if c else 3))

To do so with regular if else statements, one would need ten lines of logic. Ternarys are a lesser known function within Python, and it belongs in any programmer's set of tools.

Search and replace multi-line expressions with SED

10/26/2011 11:55pm | 2 Comments

Now here's an interesting problem: I wanted to do a recursive search and replace in unix, AND I wanted to do an expression that spans multiple lines. Here's what I came up with:

find ./ -type f | xargs sed -E -i -n
'1h;1!H;${;g;s/</fileSet>.*<fileSet>.*RevisionVersion.*
</fileSet>.*</fileSets>/</fileSet>n</fileSets>/g;p}'

There a lot of examples showing you how to do this. The first argument lists all files recursively. These are the piped to sed, which uses an inline search and replace (-i or --in-line), then using the expression '{}' which is then modified for multi-line expressions (1h;1!H;).

WebPageTest and IE9

10/25/2011 12:25am | 0 Comments

Recently, I tried updating the browser for a WebPageTest instance to IE9. This proved to have some issues, specifically due to the pop-up dialogues that IE9 has now to tell you when something suspicious occurs. Logging into WPT, I was greeted with an error on an IE9 browser opened by URLblast. Something along the lines of: "Are you sure you want to use this Non-Verified plugin?" Of course, the non-verified plugin was the WebPageTest hook. In order to get that working, I modified the security settings on my browser to not care about non-verified plugins: Internet Options (clicking on that gear icon in IE9) -> Security -> Custom Level. I modified two settings:

"Download unsigned ActiveX controls" to Enable (not secure)
"Initialize and script ActiveX controls not marked as safe for scripting" to Enable (not secure)

This then brought me to another error, with IE9 complaining about not using secure settings. Something like: "Your current settings are insecure" Well, after some searching, there's apparently a policy that you can set that disables this specific message: http://windowsconnected.com/forums/p/959/3087.aspx#3087 Basically it says: Run gpedit.msc (if you type 'gpedit.msc' in the search bar it comes up) Then Navigate to Computer Configuration -> Administrative Templates -> Windows Components -> Internet Exporer, and right click and enable the "Turn off the Security Settings Check feature" policy. This gets rid of the error, but then WebPageTest just seems to freeze on a run. After some more searching, there was one final step in the solution. It seems that urlblast has to open the browser using the user's account. By default, urlblast creates and uses a specific account on which it opens a browser, not necessarily the user that is running urlblast. Having the account opening the browser be an administrator did the trick, and in my situation, I just had it be the same account running urlblast. This can be done with a change in urlblast.ini:

Use Current Account=1

And that did it for me!

Getting Python2.5 to Build with sqlite3 and zlib on Ubuntu Natty 2.5

09/19/2011 07:21pm | 0 Comments

I had a really hard time finding this, so I'm posting it here: First one must install all the proper packages on Natty (these are the packages needed for zlib and sqlite in general, not just specifically for Python):

sudo apt-get install zlibc zlib1g zlib1g-dev
sudo apt-get install sqlite3-dev

Then one must add an LDFlag to the new lib directories (apparently Natty has a new directory for X86_64 lib files):

after the ./configure open your Makefile and find the line with
LDFLAGS =

edit to LDFLAGS = -L/usr/lib//x86_64-linux-gnu

and make

Credit for the above snippet goes to Awin Abi and source is below: http://groups.google.com/group/google-appengine/browse_thread/thread/a8bd0a71270a3ce6 Basically, setting up Python2.5 ( and presumably any version of Python) properly involves downloading the proper package libraries , then building Python2.5 with those packages. In order to do this, the LDFlags variable must have the new library location (the /usr/lib/x86-64-linux-gnu) for Natty and 64-bit processors added. I have not tried this on a 32-bit machine. This may not be required then, or you may need to point the flag to load the proper directory.

My IDE in Emacs (mainly for Python)

08/23/2011 12:26am | 1 Comments

I'm writing this article up to mainly keep track of the current state of my IDE in Emacs, how to set one up, and to keep my to-do list.

Implemented Features

Default Emacs Library Includes

I use the following from the library that comes with Emacs (as of version 23)

Viper-mode (viper-mode 3, though I'm sure 5 would be good too)
Windmove (through keybindings, for moving around windows easier)
hideshow (for code folding)
ibuffer (for listing on buffers when buffer switching)
ido (for listing of file in a directory in the minibuffer

Code to instantiate:

(setq viper-mode t)
(require 'viper)
(load-library "hideshow")
(add-hook 'python-mode-hook 'hs-minor-mode)
(require 'ido)
(ido-mode 'both)

Keybindings

(global-set-key (kbd "C-x C-l") 'windmove-right)
(global-set-key (kbd "C-x C-h") 'windmove-left)
(global-set-key (kbd "C-x C-k") 'windmove-up)
(global-set-key (kbd "C-x C-j") 'windmove-down)
(global-set-key (kbd "C-x C-;") 'hippie-expand)
(global-set-key (kbd "C-x C-g") 'find-name-dired)
(global-set-key (kbd "C-c C-t") 'ansi-term)

Viper Keybindings (in .viper)

(setq viper-expert-level '3)
(setq viper-inhibit-startup-message 't)
(setq-default indent-tabs-mode nil) ; I think this makes tabs into spaces
(setq viper-shift-width 4) ; don't touch or else...

;; Makes searching w/ regex default
(setq viper-re-search t) ; don't touch or else...

;; The following is for hideshow to work ALMOST similar to vi folding
;; (there were keybindings I didn't like)
(define-key viper-vi-global-user-map "zt" 'hs-toggle-hiding)
(define-key viper-vi-global-user-map "zM" 'hs-hide-all)
(define-key viper-vi-global-user-map "zm" 'hs-hide-block)
(define-key viper-vi-global-user-map "zR" 'hs-show-all)
(define-key viper-vi-global-user-map "zr" 'hs-show-block)

Features implemented using external files

Yasnippet (for bundling and snippets)

Yasnippet provides me features along the lives of textmates bundling, which I think definitely makes things faster in the long run. After all, who wants to write boilerplate code? http://manual.macromates.com/en/bundles Yasnippet site: http://code.google.com/p/yasnippet/

lusty-explorer.el (for a great tab completion file navigator)

Followed this emacs-fu guide: http://emacs-fu.blogspot.com/2010/07/navigating-through-files-and-buffers.html And downloaded the .el here: http://www.emacswiki.org/emacs/LustyExplorer Specifically I have the following in my .emacs:

(when (require 'lusty-explorer nil 'noerror)

  ;; overrride the normal file-opening, buffer switching
  (global-set-key (kbd "C-x C-f") 'lusty-file-explorer)
  (global-set-key (kbd "C-x b")   'lusty-buffer-explorer))

Desired features

I have yet to implement this, but I would like:

Better file search (the ones I could find don't do what I'm looking for)

Specifically, looking for a smart find that allow autocompletion
Looking for something along the lines of eclipse

Lessons Learned from the past three years

1/13/2011

Today I signed up for a new contract with my webhost, HostMonster, for another three years. To commemorate that fact, I've redesigned yusuketsutsumi.com from scratch. Needless to say, an addition three years worth of experience has served me well. Here's an example of what my old website looked like: nope

While the new site now: nope

I've obtained a lot of experience and learned a lot of lessons. I'd like to share a few in this post:

int main(){
             int i = 0;
             for(i = 0; i < 15; i++)
	     {
                 i++;
             }
             return i;
         }

Design should be elegant

When I designed my original website, all my head was filled with was thoughts regarding using the cool gimmicks seen on other websites. Things such as transparency on pages, rounded corners everywhere, and cool background images. As you can probably tell, my new site contains none of that. What I've noticed is that although these components are useful, a website does not need to use any of these to look well designed. It is possible to have a nice looking website, implementing luster only when it is necessary. In addition, fancy designs tend to only sacrifice the user experience, which is the sole purpose of a website.

Don't reinvent the wheel

I'm sure this has been hashed over again and again by others, but some people need to hear this over and over again, and experience it themselves, before it really clicks. This is what happened to me. I spent a very long time obsessed with the idea that, if I built something myself, it would be way better than anyone else's out there. And the fact is, sometimes this may be true. However, those who think like me, and actually choose do so, are not looking at the whole picture. Building anything has a much larger requirement that just the expertise to do so. It requires time, and a lot of it. One would have to spend time designing the project, laying out ever single detail of how it works, how it would talk to other services, and finally, how it handles errors. Considering all the factors, it doesn't make a lot of sense to devote so much time to one project, especially when this particular component is just one step from your own full-on service. Another hidden cost that one doesn't immediately realize is that of maintainability.

Recent Posts