Techy post

I appologise for how techy and geeky and boring this post is.

I even find it so myself.

But I need to get this off my chest, so I can actually write about interesting stuff later...

And some geeky people may actually find something comprehensible in here.  Perhaps even interesting.

If you want to read it, then if your eyes glaze over with techiness, then stop reading, drink a glass of water, scroll down to 'And another thing', and read from there.


So then.

Most of my recent work has been on the omnitube.org website.

It's a site I put together with David when I visited OMNIvision 4 years ago (now that seems weird!).  We put it together in about 2 weeks, based on a well known large complex CMS (Content Management System) called Joomla, which was the OM standard web-site-software-package, a few very large complex modules for that, specifically "VirtueMart", plus a bunch of my own hacks in php/javascript, plus a load of very clever HTML/CSS from David.  Since then, I'd done a few fixes for them, logging on from a 'net cafe in Singapore, and when Becky and I arrived at OMNIvision 2 years ago, it was basically in the same state.


The trouble was that in the 2 years since we'd put it together, the internet had changed quite a lot, and we hadn't had the time to really work on OMNItube to change with it.  Joomla had advanced significantly, as had Virtuemart, and alas, we couldn't smoothly upgrade to the latest versions of those without majorly breaking everything.


About a year ago (from now), I decided it was about time to do something about this, as I was scared that a now 3 year old install of joomla/etc without being able to have security updates was a bit scary.  Especially as we wanted to sell videos online, and DVDs, and so were doing credit-card type security stuff.  And the system admin wanted to upgrade the server, and I thought doing that would probably break everything again.


I spent about a week trying to upgrade things to the latest versions of joomla and virtuemart.  Some bits I got almost close to working, but in the end it just totally bogged me down.  Joomla + Virtuemart + all the plugins and modules which were installed to make everything work as we wanted ended up at around 7000+ files, and 800'000 lines of code.  Not joking.  So tracking down bugs and errors and trying even to find which function I was looking for took forever, and I simply don't have a big enough brain to hold that much in my head at once.

So after a week or two of working hard to try and upgrade things, I gave up, it was just too big a job for me to do on my own.  I'd made no progress, and it was still a big stinkin' mess.


We also had some new features we wanted to get working. Specifically, iTunes subscriptions, XML sitemap, facebook commenting/liking/etc, and related product lists.  The "search" system was also very broken, giving pretty much useless results most of the time.


I'd just read an article online about making stuff simpler, 'the no-framework framework' or something like that.  And looking at the SQL database, I figured I could write the iTunes and sitemap features we wanted in just pure basic php.  So I did.  It took about 3 days, which was pretty good, I thought, considering I'd not really done significant amounts of programming in about 4 years.

So then – perhaps alas? – I thought, well, why not re-write the whole of OMNItube in a similar manner?  I mean, how hard can it be?  It's a very simple website, after all....

In 2 weeks after that, I'd put together a quick 'demo' of the site, which took it's data from the joomla database, had all the basic functionality of the front end,  but in 700 lines of code, TOTAL.  No Joomla, No Virtuemart, nothing.

OK, so it was only the front end, if you wanted to actually add a new video or something you needed to edit the database manually, rather than use the horrendously over-complex somewhat-broken-by-our-CSS-mods joomla back-end, but hey!

700 lines of code is something I can hold in my head, no problem.  Way better than 800 thousand!

So anyway, that's been the big project for me, over the last year.  It's taken me 3 re-writes, which is what took so long.  I really should have spent a lot longer learning stuff and looking at alternatives before launching in to it as I did...

I first wrote it doing all the SQL in-line, sort of saying "I need a list of the products, so the SQL for that is SELECT...blah.blah.blah" and so writing it then and there.  Which works for very simple stuff.  But I began to find patterns, of course, and so wrote functions and classes to abstract stuff out, which was good!  But each time I did something like that, it meant re-factoring and writing stuff I'd already done, to make use of the new abstract methods I was coming up with.

Eventually, though, it was all working, pretty much.

 I slowly switched over bits of the live system to my new creation over time,  so the video embedding system was first, then the iTunes feeds, then the front end, eventually, and finally I was able to delete the joomla system, and run entirely on my much smaller simpler pure-php non-frameworked version.  I guess about 6 to 8 months ago?

So after the crazy summer, I did a bit of cleaning up, backups, version-management, etc, And a guy called Chris came to work with us, who is interested in doing web / programming stuff.   So I chatted with him, showed him OMNItube, as it was then, and told him about my future plans.

I'd found a cool looking database library for PHP called 'Propel', which did much of the grunt work of doing SQL database querying instead of us having to do it.  I thought it looked cool, and would save us a lot of work in the future, and so Chris started work on porting OMNItube over to using Propel.  He also figured out a bunch of stupid stuff I'd been doing before, and found much better, simpler ways to do it.  Templating things, for the big example.

In some ways, I'd not really wanted to be working on any more OMNItube stuff for a while though.. I kind of thought 'it's working, I should probably just leave it until next year or something'.  But then after Chris left, I wanted to get his project all merged in to the main system, and not leave things in a mixed state, with 2 versions, his almost-done version, and the online-one with changes to it that I'd needed to make since he started....

Anyway.  I've spent the last few weeks hacking hard at OMNItube, until now it's stable and running well using the new Propel based system.  It's much simpler, much better, the administration/back-end thing is very cool, and generally just works great.

The whole thing is now 7000 lines of PHP, plus a few hundred of javascript and html, but still, that's quite manageable, and I'm reasonably happy with it.  I've even open-sourced one of the smaller more generic bits of the system, and plan to put a few more modules online too, in the hope other people can find them useful in the future.

I've learned a lot, during this project.  I understand from experience a load of programming concepts a lot better than I used to before.  I've learned quite how much I dislike PHP, for one thing.  It really is a botch on top of a hack on top of a temporary way to fix a problem.  If I were starting the project now, I think I would pick a cleaner, simpler, better language.  Possibly ruby, python, LISP, javascript, perl5, or even haskell.

Another thing though, I'm really bored of web development.  It's so much effort, and in the end, all you're left with is a bunch of patterns of pixels on a screen.

I SO miss acting, theatre, drama, etc.

And the trouble is, after a day fighting with programming problems, I don't then feel capable of doing anything creative.  It's like my brain has just given up, as has my emotional capacity to care about creativity.  All I want to do is crash, which isn't very nice for Becky, nor very helpful in building relationships, in staying up to date with anything actually interesting, or in keeping in practice with Clarinet or juggling or writing plays or whatever else I'd prefer, if I were less frustrated and computered-out to actually do.

Writing, for instance.  When I've been writing code, or trying to write code all day, I have no ability to write on this blog, say.  Or write newletters, or emails, letters, or even SMSs.

I've not really touched OMNItube now in about a week, which is why I'm able to write this post... but even still, I'm not quite able to break my brain out of that mould enough to write about anything interesting... all I can do is write about writing code.

Which I guess is a step in the right direction.

I'll write a more general, non-techy post soon