March 05, 2015

Andrew Williams

The Version Controlled Home

For the last year or two i&aposve been using SVN to store my common configuration files, this has worked wonders and has enabled me to move my config seamlessly between systems with a few simple commands. In the last day or so i&aposve decided to move from SVN to Git as my control system, and now I think it&aposs a time to do a post on why exactly i&aposm doing this. The idea of a version controlled home directory stemmed from people storing their /etc config files in CVS, this allowed for any modifications to be tracked, tested, and if worse came to worse, rolled back without much hassle. These ideas can be very useful for the end user, imagine you want to fiddle around with your terminal settings but forget to make a backup of your original settings, the time you spend trying to fix it back can be avoid with a simple vcs command. Another situation that I often get into is when you format or move to another system, a few quick commands can return your config and no pain of trying to remember your favourite settings. So, how do we do this? Each person has their own method, and hopefully I&aposll describe my little world to everyone in a easily digestible way. It&aposs neither pretty or easy but it works for me. My SVN version was very simplistic, a single repository broke down into "packages", which contained the batch of config files for each program, such as "irssi" or "bash", these would live under "trunk" in the repository.

$ ls
abook          gtk2     ikog   keepass-private  mutt  ssh          tin
gnupg-private  hamachi  irssi  mozilla          pine  ssh-private  xchat

For each machine I made a new branch under "branches" then I would use the "externals" properties to pull in the packages that I needed from trunk. So, when I wanted to pull in my configuration I would simply checkout that machine&aposs branch into a folder then symlink the required files over as needed.

$ ls
ithaca  manex  orion  vektor

This took time to setup but once the initial linking was done it was a simple matter of managing the files in the single checkout folder. This system served me well for a year or two, but with the increase of machines and the general pain of symlinks I decided I needed a new method. I reviewed a few examples but the one that stuck with me was Martin Krafft&aposs system using Git, MR and a few handy scripts. I&aposve now managed to rework this into a similar system for myself. First of all, you need a method of getting the configuration files in the location you require. Git has this excellent feature to having detached worktrees, this allows you to remove the need for symlinking all together. For example, we can setup a git repository in a storage directory then tell the repo to checkout the files to your home directory.

$ mkdir -p ~/.dotfiles/test.git
$ cd ~/.dotfiles/test.git
$ git init --bare
$ git config core.worktree ../../

So now, you have the test.git repository, and the worktree is your home directory. Now it&aposs simply a case of checking in the files you require and commit them to the repo. This scenario is a little different from the original Git idea so a little bit of a workaround is needed to actually use the repo in this way. Two variables need to be configured for Git to use the detached worktree as desired.

$ export GIT_DIR=~/.dotfiles/test.git
$ export GIT_WORK_TREE=~/.dotfiles/test.git/../../

Now, you can use the Git command as if you were in a normal Git worktree. This is a pain to work with by hand but luckily Martin also created a little shell script to set these variables as needed. It&aposs based on zsh but i&aposm in the process to converting this to bash to avoid a extra unneeded dependency on my part. So, we can get the configuration files to the place they need to be, now we move onto actually packaging and distributing the files. I decided that Martin&aposs method works the best, using the mr tool you can configure and manage multiple repositories and automate the checkout and update of these. This with tool the management of your packages can be done by simply changing the config file of mr. mr supports importing extra configuration files based on wildcards, this allows for a global configuration to be setup which will only include configuration on a per machine basis. For example in the current .mrconfig I have this.

include = cat ~/tools/mr/lib/* ~/.mr/* 2>/dev/null || :

Simply put, this will include any files in my ~/tools/mr/lib/ and ~/.mr/ folder. Then in my .mr folder I have a file for each type of package I have available

checkout = git_fake_bare_checkout 'ssh://' 'mr.git' '../../'

checkout = git_fake_bare_checkout 'ssh://' 'bash.git' '../../'

checkout = git_fake_bare_checkout 'ssh://' 'bin.git' '../../'

checkout = git_fake_bare_checkout 'ssh://' 'ssh.git' '../../'

So when I execute the mr command this will checkout each of those repositories as needed. If I require any extra packages I can pull in another config file and drop it into the .mr folder. Now we have the method and the configuration sorted, how do we get this onto a bare machine? Again, Martin has come to the rescue in a form of a script he has setup to do the initial bootstrapping of a fresh account, it pulls in the basic configuration for mr and then it&aposs a case of dropping in the require config files into the .mr folder. Job done. While my system isn&apost perfect yet, it is workable and very flexible. The benefit of being able to move my active configuration between machines with a few commands outweighs the time needed to setup and configure the system. If you are interested my public configuration files are available via gitweb, hopefully from the mass of files you can work out what i&aposm doing. For the bootstrapping script check the setup.git repository, for my mr configuration files check mr.git. If your interested in setting up a version controlled home directory, I&aposd advise you to join the vcs-home mailing list and check out their archives and wiki. Also, remember there&aposs no all ruling version control system to use for your home directory, Git works well for me but it doesn&apost for everyone. The idea is to have a system that works for you, while I&aposve followed Martin&aposs example very closely, again, this wont fit all.

by Andrew Williams at March 05, 2015 04:24 PM

Howto: Send SMS using a Huawei E160G and Debian

People who use their Huawei E160G on Three will know that in the Windows client you can send and receive SMS, this will come at some minor cost of £0.10 per SMS, and you can add bundles onto your mobile broadband account to make this cheaper. Similar functionality can be achieved in Linux, and it&aposs very useful if your like me and want to drop someone a message when you don&apost have your phone around. For this we&aposll be using Gammu, which is a toolset for managing phones via the AT GSM command set. It was originally forked from Gnokii, which was a similar toolset for Nokia handsets. As the E160G opens a serial port with access to the AT command set this is a relatively easy tool to setup. First of all, we need to grab the packages. As these are standard Debian packages you should have no issues.

# sudo apt-get install gammu

Next, we need to configure Gammu to pickup the correct device. Check your dmesg for the serial port:

$ dmesg|grep tty
[12321.308078] usb 5-3: GSM modem (1-port) converter now attached to ttyUSB0
[12321.308275] usb 5-3: GSM modem (1-port) converter now attached to ttyUSB1

Edit ~/.gammurc, or run gammu-config to change the device settings. Your ~/.gammurc file should look similar to:

port = /dev/ttyUSB0
model =
connection = at19200
synchronizetime = yes
logfile =
logformat = nothing
use_locking =
gammuloc =

Give it a test by getting all the SMS from the device:

# gammu getallsms

This should bring back all the SMS currently stored on the stick, which should include your login details for the Three website (unless you&aposve deleted them). To send a SMS use the "sendsms" command:

$ gammu sendsms text 07874454543
Enter message text and press ^D:
Test Message!!!!!1!
Sending SMS 1/1....waiting for network answer..OK, message reference=2

Gammu has a lot more tools and options to explore, now you have the basic config you can setup a SMSD, which can expose the ability to send SMS to a network. Also, Gammu has a python interface so you can possibly build your own frontend client for sending SMS. For more details explore the Gammu Wiki.

by Andrew Williams at March 05, 2015 04:24 PM

Hacking the ZTE MF627

Its been a while since I&aposve done a good hack article. so again I&aposm back onto my favourite topic of 3G modems. Thanks to the generous promotions at 3dongles4free I&aposve been able to pickup a new Three dongle for next to nothing. As I&aposve already got my E160G I didn&apost really need this to be on the Three network. After a quick search around and a few suggestions from existing mailing lists I&aposve found out that a hacked firmware exists and these cheap and cheerful dongles can be flashed to allow any SIM card to be used. This should be a simple job of updating the software and using the new SIM card.

First of all, grab the software pack from Rapidshare, due to the questionable nature of this copy of the firmware no one has been willing to host it on their own hosting, and I&aposll keep to that idea. Extract the files from the RAR and you should have a firmware upgrade, and a installation folder for the connection software. As the existing Three connection software is very limited, the software package includes the Telstra version which allows you to define your own settings. Before you attempt the software upgrade, you need to remove any existing Three software, install the Telstra version and remove your SIM card from the dongle, then simply plug it in and run the firmware upgrade. This process will take around 15-25 minutes and once it&aposs done it&aposll give you a prompt. During the upgrade do not power off your PC or remove the dongle from the USB socket. This will brick your dongle rendering it completely useless. Now, put in your non-Three SIM card and plug it back into your PC, the Telstra software should start-up and try detect the device, you need to configure the software for your provider&aposs APN settings, but the PDF document included with the software package will give you all the details you need. Remember, I take no responsibility for people bricking their equipment, you have been warned.

by Andrew Williams at March 05, 2015 04:24 PM

Django & MultiDB to the rescue

One of the key components of Auth is the ability to communicate with numerous other systems and manage their user authentication systems in situations were we can&apost modify the application to use our authentication API. To achieve this we have the Services API, which is a generic interface designed for basic operations such as creating a user, disabling an account as so on. If we wanted to support a new system we write up a simple Python module with these functions and the API does all the required importing and abstraction out for Auth.

In previous versions of the API we handled databases in a very weird way, either by writing our own SQLAlchemy queries or boding the Django ORM to give us a basic cursor to work with, while it was far from perfect it allowed us to edit the databases of other applications without much hassle.

Recently, Django has updated into version 1.2 and with it came the MultiDB functionality which allows you to access multiple databases natively in the ORM. For our database layer this presents some new options not available to us in the old versions.

Using the Django database introspect you are able to generate a Model from a existing database schema, for this example we&aposre working with Mediawiki&aposs native database in MySQL. So first of all we need to define the database in our settings.

    'dreddit_wiki': {
        'NAME': 'wiki',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'wiki',
        'PASSWORD': 'passwordgoeshere',

Next we fire off the inspect command to produce our database layout

./ inspectdb --database=wiki >

After a short time you&aposll have a fresh Python module with all your database models nearly ready to go, first thing to do would be to edit this file and change any foreign keys to the required Django ForeignKey() field. While inspect does as much as it can it can&apost detect foreign keys.

Once your read to rock its a simple case of getting your shell out and giving it a test run.

./ shell
>>> from wiki.wikimodels import User
>>> User.objects.using('wiki').get(user_id=1).user_name

Simple! No more hassling with db cursors, just simple ORM access without the hassle. The next big leap is defining the database connections at runtime, injecting into the DATABASES variable, by doing this I can remove the problem of having to manage the services&aposs database connection in the and instead have them defined on a per service basis.

by Andrew Williams at March 05, 2015 04:24 PM

Barcamp Blackpool: A Quick Writeup

Well, i&aposm currently relaxing in my room after fantastic day at Barcamp Blackpool. I&aposve never been to a Barcamp up here so I didn&apost really know what to expect from the people of the area, because as you know the people really do make the Barcamp, but I was pleasantly surprised. Les Pounder and Lalita D&aposCruze kicked the day off, they&aposve gone to great lengths to provide a large schedule for people to talk in and even a full lunch and dinner service for the attendees. As per usual the Barcamp bugs set in when the projector failed to work after the first few slides, not like it actually affected the introduction at all.

A common thing we see now at Barcamps is community tables, accordingly this was the first time that Barcamp Blackpool has tried it but it was packed full of interesting groups and projects. A few hackspaces made a appearance with their 3D printers, showing off some of the items you can create and also some other smaller projects they&aposve been working on. As to be expected the other sponsors had a few stalls as well, packed with goodies and information about their businesses, Tim Dobson was there with the Bytemark stand who re now becoming common features in the Barcamp community (mostly due to their excellent sponsorship of UK geek events).

The first talk I stumbled into was a demo of social games on tablets, a few interesting games were demoed on the iPad, pointing out whats done right and what doesn&apost really work. I have to admit I ducked out of the session early to have a wander around the stalls, accordingly it was stacked full of a few innuendos later in the talk.

After a short break and a wander around the hotel I jumped into another talk. Concrete5 is a interesting PHP/MySQL CMS system where the actual site is the UI for updating it, much like how Google Analytic site view works you can login to a administrator mode and click and update parts of the site in a WYSIWYG fashion, with all the fluffy extras of versioning and a full publishing workflow. If you want to keep a site up to date with a simple interface to work with then this is the one. I&aposm considering using that on a new version of St. Mary&aposs website, allowing them to update the site with relatively low effort on their part.

Afterwards I stepped out into the main area to watch the "Geek Fit" talk by Kian Ryan, which was essentially a introduction lesson in fencing. I have to say it went down a storm, with about 20 people taking part and even more watching the outcome. As you can imagine people got a little excited about hitting people with plastic swords, but Kian kept everyone in line. I think a few people will be waking up with sore muscles tomorrow as it seemed like quite the workout for all involved.

Paul Brook gave a small but interesting talk on using a cheap Amatel board and flashing it as a Aurdino device, the interesting part is that these board are USB key side and provide all the outputs that you&aposd expect with the exception of the analog I/O. One attendee also mentioned that you can unsolder the USB socket easily and the size can be reduced to just a tad larger than a 50 pence piece, excellent for tiny embedded projects. The boards themselves can be picked up easily from modtraders but the interesting part is that with bulk purchasing you can get the unit cost down to £4 each. Further details can be found on the Leeds Hackspace blog.

The Shrimp is a ultra-low cost Arduino compatible board created out of the need to provide a cost effective Arduino for use in schools in deprived areas, as time goes on budgets are being slashed for schools making it difficult to justify the cost of a £16 Arduino Uno for use in the classroom, with this costing around £1.40 for the components and 86p for the breadboard to build it on it makes it affordable for schools to spend money on and give the students something to take away with them. cefn gave a live demo of constructing one of their kits and how to use the standard Aurdino IDE to program it. Thankfully he only had one minor problem when a wire came lose, but otherwise it was a perfect setup and demonstration. I highly recommend any tinkerers to check out their site and look into the project.

Afterwards the guys organised a full buffet dinner and entertainment, with Alex Martindale giving a small comedy sketch talk called "Clive Sinclare ate my childhood", then followed by Dan Lynch and 20lb Sound doing a set. I didn&apost stick it out to the end but what I heard was excellent, I sneaked off in the last few songs as being up since 6am really took it out of me.

So i&aposm all hold up in my hotel room, which is surprisingly better than I expected, the room looks like its been recently redone, well, the decorating that is as the furniture is still a little shabby, A interesting side note to end this on, this is the first post i&aposve done mostly on my Nexus 7 using a bluetooth keyboard, and surprisingly its working well, except where it refuses to switch to a English (UK) keymap. I&aposve probably missed quite a bit of what went on today, but nothing can beat actually attending these type of events and getting involved.

So on that note, roll on the next event which is Barcamp Liverpool 2012 on the 12th-13th October, quickly followed by Play Expo in Manchester&aposs EventCity on the 13th-14th October.

Oh, quick note, it was fun to find out that Norbreck Castle uses Onity for its door locks. I&aposm sure FreakyClown would of been interested in that.

by Andrew Williams at March 05, 2015 04:24 PM

Brother QL-570 and Linux

A few days ago I picked up a Brother QL-570 cheap on Amazon for the other half, as shes about to setup her own online shop and needs to throw out a few address labels. While I did check up online that it was supported on Linux I didn&apost really look into how well its supported, and unfortunantly its not good.

Brother have released a driver set for the device, but it does have quite a few issues that present massive stumbling blocks. The deb packages seem to be setup for Ubuntu only, and die horribly when installed in Debian due to CUPS using a init file of "cups" instead of "cupsys". A hacky way around it does exist, but honestly this is more poking than should be done for a simple deb package.

It seems that the driver has a few issues regarding configuration settings, define too many in CUPS and processor tool Brother includes segfaults, a fix does exist but a general lack of interest in this issue seems to perpetuate this error, even when a patch is available to resolve the issue. The driver has gone unchanged in Ubuntu and Debian for many years now.

Thankfully, it seems that Linux isn&apost the only one affected by this issue. After several hours of frustration trying to get the device to work I gave up and setup the printer in CUPS as a raw printer, allowing my Windows PCs to use it with the official drivers, only for it to happen again...

So, why the post? Well if you&aposre looking for a Linux compatible label printer i&aposd advise to stay away from Brother&aposs offering. Mine works at the moment, but its doing what I really wanted it to do.

by Andrew Williams at March 05, 2015 04:24 PM - The near perfect blogging "platform"?

For a while now i&aposve been trying to find the perfect mix of static page generation and ease of updating, my previous solution based on Git + Wok allowed for rapid updating anywhere I could get my hands on a git client, while it was perfect for working from home or my laptop it presented some problems when I wanted to do a quick "on-site" post about a event I was attending.

For the last year i&aposve been a user of Evernote, a tool to help you catalogue and organise those little things you need to keep track of. Evernote really does bring the filing cabinet with you, I can do a large amount of online research before heading off to a new country or event and then have it totally accessible on my Android phone without much hassle. Combined with Remember The Milk i&aposve got a good toolset to work with to keep track of all the little things in my life.

A few months ago a new service call went into beta, the idea is to have a blogging "platform" work from Evernote and automatically create posts on the basis of notes within your Evernote library. It was only in the last few days when I decided I should give it a go, especially with my love of simple site generators like Wok. After a few hours of usage I can safely say i&aposm never going back to the old way.

The first big positive in my book is the support of Markdown, with this simple tool I can format the pages anyway I wish and also import all of my existing Wok posts with little to no effort on my part. It seems that the Markdown they&aposre using is a little different than the one in Wok, but i&aposm working around the oddities. For the advanced users their rendering engine is built on top of Jinja2 allowing for excellent customizability which i&aposm yet to fully utilize, hopefully within the next few days I can import my existing Wok layout onto the site. also supports Discus comments and Google Analytics tie in on all their default themes along with the usual Twitter and Facebook tie ins, i&aposd love to see Google+ added to this as well but as the API for G+ is still in its infancy I don&apost expect much to happen soon.

So, if you&aposre a Evernote user wanting to setup a simple blog then I&aposd suggest using

by Andrew Williams at March 05, 2015 04:24 PM

Back to YNAB and avoiding the traps

YNAB has always been an excellent tool for me. I started using it around this time last year when I discovered the snappy Youtube videos, and the free demo. Within a few weeks I was hooked on using it, and I saw the massive advance quickly when I was able to manage my Christmas pay and put a hefty lump of money into a savings account, and all I needed was something to track my money.

After 7-8 months the sheen worn off as I was fighting with 60+ categories, a newly added credit card, and quirky statement importing, and around October I essentially gave up. From there my financial situation didn&apost get worse, but I started to slip on the credit card and with them upping my limit another £1,000 really didn&apost help matters.

So, another year, another fresh start with YNAB. Looking back I identified a few problems which really started to kick me after time and made my budget a hell of a lot more complicated;

Multiple categories for the same type of things

I found that I had multiple categories for essentially the same thing. Best example is my personal site hosting which was split down into Linode, Digital Ocean, Amazon, and Domains. I suppose splitting out is a good way to identify and budget for specific recurring bills, but for me Amazon store purchases got mixed in with my AWS charges and it started getting messy quickly.

So this time round i&aposm having a generic "Hosting" category to budget into, £15 a month should cover all my costs and also save up some money for those 2-3 year domain renewals. Sticking to one category should hopefully make my money a lot easier to manage in this area.

Make use of split transactions

For a long time I would do our weekly shop at Tesco and end up picking something up for the house, either new bulbs, paint brushes, or a duvet. The problem is I never sat down and split out the transactions into the food shopping and the house budget, which usually ended up with the food category taking a beating all the time while the various smaller categories never get touch, and end up being raided to make up the shortfall in other areas.

I suppose this was compounded by split transactions being unavailable in the mobile client up until the middle of last year, by which point I was already in a bad habit and wouldn&apost change so easily. The lesson here is that splitting your transactions will pay off, even if it does feel like you&aposre wasting time doing them.

Simplify your categories

At the end I had 72 categories for my spending, as you can imagine trying to fill out the budget on payday took some time. I spent so much time flicking through my categories and assigning £10/£20 here and there that by the time I got down to my savings accounts I was putting in around 10-20% of what I originally was doing in January. Because I had a budget for damn near everything it seemed like I could validate spending it, on new techy toys and vaping gear.

At the restart i&aposve settled at about 20 categories that seem to represent the core spending in my life. I have the bonus that my girlfriend takes care the household bills from the joint account, which simplifies my view as a single payment to the account on a monthly basis. If you get too far down into the categories then you&aposll be in danger of validating any type of spending, just as I did.

Reconcile your accounts

At first I never bothered using the reconcile function within YNAB as my totals always "worked out". After a few months of using it I discovered strange situations where the total amount of budget money didn&apost match up to the total available, its a common problem with YNAB and its heavily covered in various forum posts.

One of the biggest tools to assist you with any oddities is reconciliation. Grab your statement and sit down with YNAB and comb through your statements, its a great way to pick out that incorrectly categorized purchase or the missing 53p you can&apost find anywhere.

Also, if your bank offers the feature, import and match your bank records. It takes a few minutes to do and can quickly pick out oddities. Also it has the bonus of marking items as "Cleared" so showing you within YNAB your cleared balance, which is handy if you&aposre out and about with just your phone.

Use the community

YNAB has a large and popular forum community, make use of it to answer your questions or generally get support with your financial situation. While its mostly keyed towards the American audience, they&aposll help you with YNAB. In the UK i&aposd suggest using Money Saving Expert for asking any financial question without paying for legal or accounting support.

So, I hope this has been helpful to you, i&aposll keep you posted with how I do in 2014.

by Andrew Williams at March 05, 2015 04:24 PM

Goals & Aims - 2014

First of all, I am not a big new year resolutions person. In the back of mind I usually have a few things ticking over about what i&aposm going to do "better" this year, and this is probably the first time i&aposve took some time to try and jot them down and have something to look back at and decide if it went well or not.

For this i&aposll be splitting down my goals into 4 easy areas, Professional, Personal, Financial, and Leisure.


My day job has changed for the better, in a recent reshuffle my skills have been moved officially under the "Development" branch of the company, and with it a job title to suite, LAE Developer. For a long time i&aposve been looking for the infamous "developer" title to help my career, and now its time to use it and start prepping myself to move around a bit.

  • Take time to re-learn Java in a business development environment.
  • Work more on my Python skills, start working with other frameworks out there (Flask, Pyramid).
  • Read about Agile development methodologies.
  • Actually start, and stick to a project using test based development.


I&aposve never been Adonis, and i&aposll never be, but its time to start taking care of myself as I can&apost just shrug it off like when I was younger. In previous jobs I had the advantage of walking to and back from work, along with heavy lifting most of the day so it worked against the junk food I was shoveling into my throat.

  • Switch to a keto diet, largely reduce my carb intake.
  • Take up some sort of regular activity to assist with weight loss.
  • No arbitrary number for weight loss, just a aim to lose weight.


In 2013 I started well by using YNAB to get my finances in order. By October I suddenly hit a mental block after I spent 3-4 hours fighting with a £3 error that seemed to link a lot of irregularities in other budget balances. Since then i&aposve not touch YNAB and while in general my financial situation hasn&apost changed drastically I really need to get everything back into check again as somehow a £1,000 credit card bill snuck up on me in the new year.

In addition to this, we&aposre saving for a house deposit, and while in 2013 we made massive strides towards it, we&aposre still only at 20% of what we need for our dream house.

  • Restart using YNAB, as soon as possible.
  • Rebuild my £1,500 emergency fund.
  • Get rid of the £1,000 credit card debt.
  • Work to get the housing deposit up to 80% of what we need.


While the other areas had just a few notes of overall goals, this will be a larger list. While myself and Jo have plans for saving and such in our mind a priority to us is to get out there and see the world and experience new things. So theses are not really goals in the strict sense, but these are items we&aposre aiming to do sometime this year.

  • More European cities; Oslo, Copenhagen, Berlin
  • Camping in Wales, maybe Yorkshire again.
  • Attend either the British or Belgium Formula 1 grand prix.
  • Summer holiday in Iceland.
  • Eurovision 2014 in Copenhagen.

So, that sums it up. By the end of 2014 i&aposll also be doing a yearly review, including some personal metrics. Also as the year goes on i&aposll be posting about any interesting things that happen and my progress in general. But for now, carry on as usual.

by Andrew Williams at March 05, 2015 04:24 PM

Deluge Web Interface and Ngnix

After a short, frustrating time, i&aposve finally got the Deluge WebUI to proxy through Nginx without any errors. The revelation came when digging through the Deluge forums I found a little nugget of information which solved it all, a small header call X-Deluge-Base that when passed will prefix any media calls made in the page with that text. So instead of setting up weird aliases and fiddling around with Nginx&aposs options to get it to work I could just specify that and use a very basic server config.

upstream deluge  {
  server localhost:8112;
server {
  server_name  deluge.home;
  location / {
    proxy_pass  http://deluge;
    proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host            $host;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Deluge-Base   "/";

by Andrew Williams at March 05, 2015 04:24 PM

Doxie One Scanner

As some may know, I have a utter deep hatred for all things Epson now. Around about 3 months ago I went to print something and the printer delightfully informed me that my Pink was low and that I should replace it, never mind that I was printing a letter purely in black and white, it still wanted the pink. We didn&apost have a cartridge to spare so I had to go get one from the shop, thinking that all the other levels were ok. When I got back I was frustrated when I switched it out and the printer ended up prompting for cyan as well.

Needless to say, its now sat in the junk pile and a nice cheap laser printer has replaced it.

...but this isn&apost what this post is about. The real frustrating part of using a multifunction Epson printer is that you can&apost use any other functions of it unless the ink cartridges are ok, which really kicked me in the arse as I used the printer to scan in all my documents into Evernote. So now I had to find a replacement.

I had a look through the market, and the amazing expensive but excellent "Evernote Scanner" which is a rebranded Fujitsu iX500, and some dirty cheap flat beds from eBay but one did catch my eye;

Doxie One

The Doxie One. Available for £99 (at the time of writing) on its a small and compact scanner that can operate without a PC, dumping the resulting files on a SD card in a standard DCIM file layout, so anything that can read a camera&aposs SD card can take these images. Excellent.

The Doxie series of scanners are made to be small and useful, its bigger brother the Doxie Go also adds a higher scanning resolution (600 DPI) and a internal lithium-ion battery. This isn&apost to say the One is inferior, it has a 300 DPI resolution and takes AAA batteries to just as portable as the Go.

So the scanner itself works wonderfully, it chewed through my 40 page tenancy agreement with ease and the software allowed me to clean up the pages and "staple" them all together as a nice single PDF to store in Evernote, while it doesn&apost win any awards for image quality (after all its 300 DPI) the speed and portability of the device really does make up for.

If you need a small, portable scanner to plow through your odd documents and receipts ready to be stored in Evernote or its kin, its an excellent buy and i&aposd suggest it.

by Andrew Williams at March 05, 2015 04:24 PM

The Big Purchase

For what feels like forever i’ve been agonising over which laptop to buy to provide me with a decent, portable development environment. I dislike being stuck at a desk, for long periods i’ve always had a decent machine to sit on my lap and pound away at when the ideas come to me, in a previous life it was my tiny Eee PC, and before that a 12" Powerbook. For the last four or so years most of my code has been wrote on a i5 home build PC running Windows, and now that the machine is a little long in the tooth and starting to drag I decided it was time to change.

So today I went to pickup a 13" Macbook Pro, middle of the line one with 8GB RAM and a 256GB SSD, the old PC will be retired to gaming and the Macbook will be my go-to machine for general browsing and development work.

I’m not a “cult of Mac" member, I’ve had the odd Apple devices in my past but theres no way in hell i’m going to drop my Android phones for iOS devices. For me, Apple provide some excellent hardware that can’t really be equaled at the moment, and the “Apple Tax" has slowly disappeared since the switch to Intel hardware they present themselves as a well price device in the upper mid range laptop market. I had a look through the various competitors over the last few days and while they produce hardware around about the same specs and price as a Macbook it seems theres always concessions, better CPU, lower spec GPU, lower price, lower quality screen.

I’ve also come to realise that the majority of my workflow (Evernote, RTM, YNAB) have all OSX versions with equal or better quality, the odd game I enjoy playing on the go have equal versions on OSX as well (Prison Architect, FTL). So for me the Macbook sneaked in as a choice. I know some people will berate me, but hell its just a laptop, and it works for me.

by Andrew Williams at March 05, 2015 04:24 PM

New Project Woes

Finding a new project is hard. Very hard.

Since i've stopped playing EVE Online i've found it very difficult to come with new development ideas. In recent months i've padded my time with some smaller projects and ideas but usually lost my drive behind them within a few weeks. So what to do?

This weekend i've been thinking of a few things that slow down my workflow, either time tracking, task planning or something to that nature. For a long time i've used Dave Seah's Emergent Task Timer to track my time usage in work, for a while I tried the Flash app but it didn't really worked how I wanted it. After a while I ended up going offline with Dave's excellent templates and i've been a happy user for well over three months.

So the idea popped into my head to make a persistent, online ETT that allows for some quick exporting into a format I can use for my timesheets in work, but after a while I decided how much time would I have to spend to improve the paper system that Dave has honed over a long time? Especially now that he has expanded onto Amazon and selling pre-printed notepads of his ETP sheet.

The next idea was to create a CLI for Trello, and after about five minutes I realised its already a heavily done solution, fine no Go or Python client, but still done quite a bit.

So here I am, twiddling my thumbs, waiting for the next idea to pop into my head.

by Andrew Williams at March 05, 2015 04:24 PM

How not to sell an item

More of just a follow on from my previous post, I remembered quite a little amusing nugget that happened last night.

I was picking up the Macbook Pro and I was eyeing through the accessories for a spare power adapter for the home office, saving me time moving it around with me, an employee approached me and asked the usual questions, after I explained I was picking up a Macbook Pro and a extra power cable he started telling me about the Superdrive (paraphrasing what he said);

"If you&aposre looking to get a Macbook its an essential item, i&aposve got one with mine, i&aposve probably only used it once in about 6 months, but you can&apost be without it."

Not really the stellar use-case I was looking for, and not really something you should say if you&aposre trying to upsell. I do get what he was aiming for, i&aposve got a small Samsung portable DVD drive that is USB powered and it really came into its own when I realized not a single PC in the house had a optical drive, but still, he could of worded it a little better.

Anyway, if I wanted a £65 super drive to reinstall OSX or something I can grab a Thunderbolt to Firewire adapter and use my 12" Powerbook in target mode, Ooooh yes.

by Andrew Williams at March 05, 2015 04:24 PM

Always catch errors

On the 31st of Jan my NAS stopped responding, no idea what was going on and with zero response to the power button I did a hard reset, I spent the next few hours double checking all my config to find out what the hell had happened. I couldn’t find a solid reason, but at least none of the hardware was failing which give me some good news, I marked it down as an odd issue and carried on.

The same happened tonight, exact same result but this time I was prepared to some point. After attempting to login in the console and seeing memory allocation errors, then SSH dying on its arse, I checked my Munin install and notice the memory was heavy swapping. This machine has about 8GB of RAM but at any time its using about 600mb, at first I thought it was a memory leak in something but usually OOM Killer does a good job smiting any unruly processes. Then I checked my process list and noticed it was well over 4,000 sleeping processes, something had obviously gone wrong.

On my Deluge setup, due to the instability of a few of the trackers I use, I have a small Python script that checks the current state of the torrent and if they’re “red" it restarts them. Deluge’s API uses the Twisted framework to make everything async and accordingly a lot easier to work with, this was my first venture into the land of Twisted and it seems I made an error; I didn’t catch the “unable to connect" error. So after it was unable to connect the Twisted reactor was sitting there and running constantly, and as this job was running every 5 minutes it stacked up over 24 hours and killed the machine.

So, its always worth checking for errors, and not assuming that it’ll sort itself out. Lesson learned.

by Andrew Williams at March 05, 2015 04:24 PM

Python Packaging, the right way

Last night I spent a hour or so packaging up some Python I made to scratch an itch into a distributable module. Packaging has never been my strong point and I always ended up making a fiddly that had some minor problems or didn&apost work as expected. This time was especially noteworthy as I had a product that was both Python 2.7 and Python 3.3 compatible.

Thankfully Jeff Knupp posted about open sourcing a python project the right way, which covers getting your project setup right, making it easily testable, and getting it working on TravisCI.

So, my project is live on GitHub, is it useful? Probably not, but at least i&aposve put it out there incase people want to use it or improve on it. Next on the todo list is some better documentation and more tests.

by Andrew Williams at March 05, 2015 04:24 PM

Easy bag organisation with the "Grid-All"


I’ve been eyeing up the Cocoon Grid-It for some time, it looked like the perfect tool to keep all that loose crap in my bag organised in a sensible and easy accessible way. The only thing that pushed me away was the price which just seemed a tiny too high for the actual product itself, I understand its quite a unique idea and they put a tax on that, but still for a stiff piece of card with some woven elastic over it?

Thankfully the other day I came across a Chinese “competitor" version, a little bit cheaper and into the price range I was looking for, so I snapped one up on Amazon and gave it a go. When I received the package yesterday I was a little shocked that it came in the exact same packaging as the Cocoon versions you see in the Apple store. It seems that Cocoon has suffer the same fate as other companies, their unique product designs are re-badged by the factory that produces them for their own direct cut of the product. The most common victims of this are the electronic cigarette community where frequently you’ll have clones or the actual mass produced products rebadged and sold at a large discount over the original creators. Its a shame to support these type of tactics but I originally thought it would be a simple competitor instead of a direct factory rebadge and rip-off.

The product works exactly as its designed, holding tight even small USB OTG cables, and the rubber threading through the elastic keeps good firm grip on any items you put in it. I picked up the 30cm x 20cm version which is large enough to hold the bits I need on the go, and it easily fits into my Crumpler laptop bag with lots of room to spare.

So, i’d highly recommend it, but get one from Cocoon, and support the original innovators and not the knock-offs.

by Andrew Williams at March 05, 2015 04:24 PM

The Grand Reorganisation

I've been an Evernote user for a few years now, and its really changed how I keep track of "stuff" I need to keep stored for the future, be it scanned copies of my rental contract for my house or that 5 line how-to guide I found to fix an annoying issue I have. I don't think I could last a week without checking in with the reference i've built up over time.

I'm by no means a heavy user, my Evernote store consists of around 350 well curaited notes, most of which have a large PDF scan or image attached. My notebook collection has evolved nautrally with my usage and its ended up with around 50 notebooks for my very meager number of notes, so it was time to simplify.

The last day or so i've been reading a few blog posts from various Evernote users, from the heavy hitters down to the people who use it as a scratch pad for ideas. What I found is that a lot of people seemed to share the same consistent view; split down the important bits into notebooks, the rest can live in one "filing cabinet" size notebook as long as they're well tagged. Jamie Rubin documented his notebook reorganisation and his method really made sense to me. Afer around of poking around I had the following layout:

  • Personal
    • Medical
    • Banking
    • Documents
  • Professional
    • Certifications
    • HR
    • Job 1
    • Freelance
    • Payslips
    • P60s
  • Reference
    • Contracts
    • Receipts & Warranties
    • Repairs & Service
    • Filing Cabinet
  • Travel
    • Europe
    • North Ameria
  • Shared
    • Public Notes

I still have some crossover between the notebooks, but its a lot better than the previous 50, i'll have to let this bed in for the next few weeks to see how this works, but i'm already experiencing the advantages of having a single "Filing Cabinet" notebook with all my miscellaneous clippings in. I'm sure in the near future i'll have to start mass tagging items instead of depending on flicking through the list of notes, but thats for another post...

by Andrew Williams at March 05, 2015 04:24 PM

Open Sourcing Past Projects

Over a year ago now I stepped back from being a system administrator and developer for the EVE Online alliance Test Alliance Please Ignore, part of my role there I spent hundreds of hours developing their internal authentication system and other small applications that hang off it. At the time it was quite unique and only a handful of other alliances had that level of technical setup. So as you would expect like a small company with something to lose the code was buried away on private servers and rarely looked over by new people.

Today is a very different place, Auth was created at the start of a open source revolution for EVE Online applications, and over time more and more have started becoming open with now specific projects being spun up (such as ECM) to create tools, or large alliances (Brave Newbies) opening their backend for everyone to use.

The repository copies of the code I have a quite out of date, and i'm purely the copyright holder of them, which gives me the power to license and open them as I see fit. Now that its been a good amount of time since I left I feel I can safely release these into the public domain without doing any disservice to TEST and the current sysadmins.

So over the next few days i'll be looking to move the following repositories from my private Bitbucket over onto GitHub:

All in various states, but hopefully useful for someone.

[Update - 2014/03/13]

Also i'll update with links once they're over.

by Andrew Williams at March 05, 2015 04:24 PM

Service Shutdown

As some of you may know, i’m getting married, and with that comes the large cost of the actual wedding day itself. With the price creeping up with every wedding fair we go to (it seems, bitches love chocolate fountains) so I have a mighty need to save as much money as possible. With that I can’t really warrant the £70/month hosting bill i’m having to pay out for various freebie services I offer out to my fellow TEST members and other stuff in general. So the following services will be shutdown:

  • EVEThing
  • Testcursions
  • Scantool /
  • Meowbet
  • Dropbot
  • Queuebot

For the apps that have been custom build for EVE stuff, i’ll organise open sourcing them in a few weeks.

by Andrew Williams at March 05, 2015 04:24 PM

February 06, 2015