AIAIO: Our Blog

Magento is great, but it needs a good amount of hardware behind it.  Developing locally can get slow and cumbersome unless your environment is tweaked properly.  Here are a few tips for boosting Magento performance without impacting the rest of your development environment.  Please keep in mind that the memory allocations work well for my machine (dual core, 4 gigs of ram).

Database
Install innoDB.  Magento can use the in-memory buffer pool to cache table indexes and data.

Configure my.ini:
innodb_buffer_pool_size  = 64M
innodb_thread_concurrency = 4 (or 8 if you have dual core)
query_cache_size = 64M
query_cache_limit  = 2M

apache
enable mod_expires in httpd.conf

php
in php.ini enable:
realpath_cache_size = 16k
realpath_cache_ttl = 120

Install the eAccelerator binaries for php.  APC is a better solution but is less compatible with windows.  If you need to compile these, click here for instructions. Then configure it:
extension=eaccelerator.dll
eaccelerator.shm_size=64
eaccelerator.cache_dir=C:\PHP\tmp
eaccelerator.enable=1
eaccelerator.optimizer=1
eaccelerator.check_mtime=1
eaccelerator.shm_max=0

Install memcached.
add the following lines inside the config of epp/etc/local.xml
<cache>
    <backend>memcached</backend>
    <memcached>
        <servers>
            <server>
                <host><![CDATA[localhost]]></host>
                <port><![CDATA[11211]]></port>
                <persistent><![CDATA[1]]></persistent>
            </server>
        </servers>
        <compression><![CDATA[0]]></compression>
        <cache_dir><![CDATA[]]></cache_dir>
        <hashed_directory_level><![CDATA[]]></hashed_directory_level>
        <hashed_directory_umask><![CDATA[]]></hashed_directory_umask>
        <file_name_prefix><![CDATA[]]></file_name_prefix>
    </memcached>
</cache>


Admin Backend

• Keep the indexes up to date (System > index management)
• Compile Mage classes (System > tools > Complilation)
• Enable all cachine (System > Cache Management)

Ai braves the NYC blizzard to grab some Shake Shack, and even got some free hot chocolate!

One last one before the new year.

Our VP of Security takes on an 8 legged intruder.

I was adding on an app to a Django project at work where I was overriding an existing template but did not have access to the view that called that template. I was left in a scenario where I had the variables that the view was originally set up with, but non of the new models that I had added.

In a filter you can do whatever logic you want, and then pass information back to the view. Please keep in mind, this is probably a horrible practice, but it does have its uses. In this specific scenario I needed to query the new models without modifying the existing view, solution: add a filter and do the querying there.

This is the filter that I used to do the querying:

from django import template
from stager.jira.models import JiraProject, ProjectLink
from stager.staging.models import *
register = template.Library()
def has_jira(value, arg):
client = Client.objects.get(path=value)
project = client.projects.get(path=arg)
try:
jiras = ProjectLink.objects.get(ClientProject=project).JiraProject.exclude(filter_id='')
return True
except:
return False
register.filter('has_jira', has_jira)

Then, in my template:

{% load has_jira %}
{% if client.path|has_jira:project.path %}
Jira

{% endif %}

A more general example if this would be to work around the annoyance of not being able to have multiple tests in an if statement in a template: You can’t do {% if this and that %}
A solution would be:

def if_and(value, arg):
if value and arg:
return True
else:
return False
def if_or(value, arg):
if value or arg:
return True
else:
return False

 {% if True|if_and:False %}
show
{% else %}
don't show
{% endif %}

Let me know your thoughts, pros/cons of this method.

Ai’s stager project is open source and can be found at github.

Ai’s David Napolitan and Tim Broder fix a late night production bug.

About a year ago I wrote a webapp called twitter2gtalk, which simply takes your latest Twitter message and sets it as your Google Talk Status. Shortly there after I attended an App Engine Hackathon (GAE) at Google NYC. At the hackathon, I started porting the app over to app engine. At the time, there was no scheduling or cron support in GAE. I created a hybrid app running between GAE and my webfaction server. My webfaction server would handle the cron, proxy out the requests, and do some of the longer running tasks (the limit on an app engine request is 20 seconds).

Now with the release of the Task Scheduler and Task Queues, you can break up a long winded process into the cloud. My batch was taking around 18 minutes to run, with about 800 users being processed each time. For each user the app had to get their Twitter status, then connect to Google’s xmpp server, and after a couple of xmpp calls, update the Google Talk Status. Even being single threaded, this whole process started eating up a lot of juice on the server. The rest of my apache processes suffered and the entire site slowed down

Using the new app engine features, I was able to do two things: 1) Move the scheduling into app engine itself and 2) have app engine do the bulk of the work, by having each user-update be its own task. These worker tasks are then processed anywhere in the cloud, and can be run in parallel, independently of each other.

The original code was something like:

for users in all my users:
get twitter status
connect to gtalk
get current status
update status

My process for converting this to app engine tasks was as follows:

1. Create a process to load all of the users into a Task Queue
2. Set up each Task Queue so that it can independently do the work for the user that it is associated with
3. Schedule Step 1 to run every half hour

Skeleton Code for Step 1:

class TaskLoader(BaseRequestHandler):
def get(self):
logging.info(“Starting to load tasks %s” % datetime.datetime.now())
users = Account.gql(‘WHERE active = :1′, True)
count = 0
for user in users:
send_key = # data to send to the worker
taskqueue.add(url=’/worker/’, params={‘key’: send_key})
count += 1
logging.info(“Ended load tasks (%d users) %s” % (count, datetime.datetime.now()))

The Url call for this method: (‘/taskloader/’, TaskLoader),

The Skeleton code for step 2:

class TaskWorker(BaseRequestHandler):
def post(self):
key = self.request.get(‘key’)
t = ”.join(["http://django.gpowered.net/xmppproxy/", key])
logging.info(“URL !%s!” % t)
result = urlfetch.fetch(t,
None,
urlfetch.GET,
{‘Cache-Control’:'no-cache,max-age=0′, ‘Pragma’:'no-cache’})

Url call for this method: (‘/worker/’, TaskWorker),

So, TaskLoader loads all of the users into the Task queue. App Engine processes these when it has the cpu cycles to do so, and for each of these, TaskWorker is called

I can schedule these using cron.yaml in my project:
cron:
- description: load the task queue
url: /taskloader/

schedule: every 30 minutes

This is a general overview of how to break up your batch in to smaller, easier to manage tasks. Please see the App Engine documentation for more detailed information.

With the release of Android 2.0 on the Motorola Droid, the Google Maps for Mobile application brings turn by turn GPS navigation (with voice) to your car. A great feature by far, it actually made Garmin and TomTom’s stocks tank, but what does it mean for the future?

Currently adwords is all over the web; on pages, in your mail, on maps. Now picture this possibility and keep in mind this is all hypothetical: You are using Navigator to get you to the local movie theater and it randomly chimes in with “You are about to pass Joe’s Pizza!” This could be good and bad. Good because you might be able to set the navigator to find restaurants you’ve never tried and didn’t know were there. Bad because it could get annoying. Good because it would open up a whole new form of advertising for small businesses.

Either way, just something to think about, times are changing with all the technology we now carry around in our pockets.

A demo of navigator in action:

If you aren’t already familiar with Google Wave, check out some basic info, and a very in-depth video here. Wave is a new project from Google that reinvents email communication. Forget everything you know about email, it was invented back in the 70′s, things have changed; technology is faster, we have cloud computing, web apps look fancier, and for a while, a large portion of email users are moving (back) to web based clients.

Email is the current method of communication and collaboration when working on a project. You usually have a folder for that project in Outlook, some rules to filter project related messages into that folder, and an email chain for each issue. People reply to messages inside that chain somewhere, the thread gets continued, people get added to the thread, people drop off. You don’t know where you are in the chain when you check in a few hours later, and you get bombarded by… STOP!

Wave is very early in its adoption, it’s invite only, but it has the strong potential to fix a lot of these problems. It can clean up a lot of the clutter of project communication and throw it on the cloud so you can get to it anywhere. Wave does need some more security built in for the corporate settings, but that is in the pipeline. Right now waves can only be private or public, but once inside a private wave, that user can invite anyone.

Some notes on how Wave can help a project:

• Each Project would get its own folder in wave
• Anyone working on the project would have access to this folder
• Every issue or conversation would get its own wave and everyone who needs to be involved in the issue is added to the wave.
• At this point every project related message is confined to the projects folder. No Outlook rules or message dragging will ever be needed. If a new person needs to be involved in the discussion, they are just added to the wave, no forwarding or reply-all.
• The conversation can continue similar to email, with individual replies, but it can be so much more. In line replies with related topics can appear right with the original topic, not hidden down 6 replies in an email chain
• You can show only new replies on the wave and get caught back up in the discussion quickly
• You can do a playback of all or part of the discussion and see who chimed in and when
• Need to share documents? You will eventually be able to drag them right out of your file system into the browser and into the wave. (This is currently only supported for pictures)

As I said, Wave is very early in its adoption, but after more people join, and more developer plugins come out, I think it will be a very valuable tool both in the workplace and at home.