Eclipse Eclipsed?

I have found a worthy adversary to Eclipse: NetBeans. Its PHP support is... I guess "heartwarming" would be a suitable word. It's even more Java-In-Your-Face than Eclipse, but I guess it's not a surprise, when put in the context that both IDE:s are kind of a marketing channel for Java. I guess there's some kind of a piss-fight between Eclipse and NetBeans going on - ironic, as NetBeans is based on Eclipse.

A very quick review based on a few minutes of playing around: It feels nice, although a bit rough to the edges. Eclipse seems clumsier and NetBeans more up-to-date. But on my Mac NetBeans seems inexplicably slow. It's not chewing on my CPU constantly, but the lag on most controls is mind-boggling (in the 1-second range). NetBeans has many things integrated that need to get installed as plug-ins for Eclipse (SVN, SQL support and SQL query dialogue(!), transparent uploading/copying of files to appropriate web servers via local filesystem or FTP).

Talking about plug-ins, the Eclipse plug-in system has always been more or less a laugh. I've compared it to the early RPM-craze with RedHat: endless dependency recursions, and you have to hunt them down mostly by hand. NetBeans has a working central repository of plug-ins, which just install on the computer and simply work.

Other than the mysterious sluggishness, I have one major gripe with NetBeans (thus far): There's too much chrome. I'm working on a MacBook Pro with a 15" screen and a code font size of 10 points (barely tolerable), and I still have way too little workspace for the code. While I do admit that 15" is not too much to begin with, but Eclipse's not-at-all-too-minimalistic GUI looks spacious when compared to NetBeans'.

If I can get to the root of the sluggishness of NetBeans, I most probably work with that for a while, to see if it would be a replacement to Eclipse.

In a nutshell: I get good vibes out of NetBeans and its young age shows in both positive and negative ways.

Update: I seem to have found a tweak that's plaguing more than me or even NetBeans. I needed to add the line "-J-Dsun.java2d.ddoffscreen=false \" to NetBeans.app/Contents/Resources/NetBeans/bin/netbeans on line 185, under "eval launchNbexec \". Smoother, but some actions (like tab reorganization) still take weird amounts of time.

Eclipse/Subclipse Is a Handful

GAARH.

A free (as in speech and beer, and beer I would need right now) tip for y'all: If you have an entry in SVN (and use subversion to handle that) and want to redo the whole file/directory and still have a backup of it, do not - and I repeat NOT - rename the old file/directory and try to create a new, blank, one instead. Subclipse gets very angry of this, and everything goes awry.

Yes, I figure I should've done a branch/tag and done it that way, but it didn't hit me until I saw the aftermath. It's ugly. Believe me. Don't do that.

Now, back to repairing stuff to the way they were...

Uh-Oh

I might be forced to rewrite a big portion (if not whole) of the template engine.

Currently, the templates are processed in the following order:

1. Comments are removed from top to bottom


2. Variables and their filters are evaluated from top to bottom


3. Tags are evaluated from deepest nesting and top to bottom.

This works just fine if you have simple tags, doing simple things. I even got an if-then-else construction working, which I'm somewhat proud of (i.e. I'm proud that I got it working - the implementation is a matter on its own). But now I'm trying to get a foreach-construction working, and I'm knee deep in The Stuff.

Things shouldn't be all too hard to get working just well with consecutive foreach-constructs. Despite the fact that variables are evaluated into plaintext before tags are processes, I could get the callback variable working pretty easily from inside the foreach tag. But the problems start when foreach tags would be nested, which wouldn't be all too uncommon (think tables with unknown dimensions).

The problem lies in the deep-up process order of the tags. Normally, when doing nested foreach-constructs, we utilize the first (most shallow) foreach clause to get data into the child (deeper) foreach. But with this processing order, the ordering gets backwards and counter-intuitive. And I won't have anything counter-intuitive in this project.

So, I have two choices: Rewrite the template engine or skip more complicated (thus more useful) tags. Frankly, I would hate do either.

PS: Seems like the ordered list broke Google's stylesheet. It's always heartwarming to find bugs in other people's doings :)

Ignorance is Bliss

I'm still avoiding the Big Biggie - many-to-many relationships. At least I fear it's a Big Biggie. It could be a straightforward wrapping of a few ForeignKeyFields. But I'm still avoiding it, because it probably is a big horror show waiting to happen.

Case in point, just now, I started fixing on the very first rows of code in LightFrame. And it's good that I did, because the logic was totally wrong, so that's fixed now. Then I started mucking about in the SQL object, which also was a good thing, as now (I think) I have a working mysql_last_id() counterpart for PostgreSQL (why, oh why, doesn't pgsql have it natively implemented?!).

Anyhow. I thought that this meta-development, or development for development's sake, can take a rest for a while. I've decided to try to do something real with LF. Something simple, but practical - like a blog of sorts. Whatever I find lacking in the current LF implementation, I will code it as I go. This maybe means that I start working on some magic, built-in, CRUD, which should be very useful.

Once I've got the simple blog written down, I have a project that has actual use in the future. The project has actually been on hold (for months), waiting on LF, so that I could rewrite it simpler. It's not going to be public, but the complexity of it is somewhere in the blog-area, so it's not something awesome. Merely useful, needed and something actually in use. I'll see if I can get the testblog hosted somewhere for everyone to tweak on. Perhaps I'll get it stress tested also, which would be very interesting to see. I'm expecting mind-numbing slowness, but at least I can see where the clogs are, although I have my good suspicions already where they are going to be.

Anyhow. Little progress, some fixing, big plans. The usual. The progression will (or at least should) slow down even more due to a few thousand photos waiting to be edited, some job interviews and two dreaded exams. I actually should be reading to them right now, instead of writing here. But, obviously, I'm not.

Revision 100!

Open the Champagne, pop the Cognac, light the Cuban - it's party time.

Actually, the number is quite arbitrary, as Google Code includes each wiki change to the SVN count. This is actually quite weird, since issues are still stored outside SVN. In actuality, I've hit revision 100 a good while ago, because I had a revision ~80 on my home computer before coming over to Google Code, and didn't really need to export the repository.

While on the topic of hitting arbitrary milestones, LightFrame has grown past its 3000:th line of documented code.

But I still give anyone reading this blog permission to take a celebratory beer, cider, long drink or equivalent.

Progress Report:
I think I've got working the thing about retrieving data with criteria through a foreign key (as described in Increasing Complexity) - in short, making JOINs and doing WHEREs by the other table's columns. I say "I think", because I can't think of any more places to fix in order to make the required changes to work with all affected parts. That, in my book, is not a very good indicator of completeness. I'll hit those bugs not much later on. I just know it.

There's a few places that need to be optimized, but I'll leave that for later, because I still adhere to the "Get It Work Now, Make It Smart Later" (GIWNMISL!) mentality. Once I get that abbreviation looking a bit smarter, I swear I'll develop it and market it as an Agile Method, publish a book about it and become a millionaire-by-consulting.

Next:
Many-to-many relationships. I can hardly wait for the troubles I'll get into with that. In fact, the path should be set by now with foreign keys finished and all, but it's never that easy, is it? After that, the temptation is great to continue within the domain with i18n, but I fear I have to turn my attention to continue the work on maintenance scripts. *sigh*

Digging Deep

You know you are using something at its limits when something widely used, known and supported starts to behave erratically. I just hit a bug in PHP and boy was it a chore to debug and trace it back to that.

Just a FYI...

Recess!

I haven't done much to LightFrame the past days, and probably won't touch it again until Monday-ish. It's not that I would've hit a wall. Actually, stuff seems a bit uncomplicated at the moment (and I'm doing the best I can to savor the moment). No, I've been busy getting going to job interviews (who would've thought, LightFrame might be helping me out already), reading to exams, taking exams... you know. "Life." Tomorrow I'm dispatching myself some 150km off to photographing an anniversary party until the late night, which I'm quite excited about.

But, yeah. Monday come, and I probably get something done, until more "life" interferes.

Oh crap - this has become my personal blog, not something I intended originally. Well, since I'm the only one doing this, I guess it's OK. Once I get helpers, I'll get more serious, less personal. My word on it.

Not-Propel

Since comparisons to existing products fail massively, I'll correct some things:

- I'm not doing a variant of Propel. I'm doing ORM.
- I'm not doign a variant of Django. I'm doing a UMTV (URL, Model, Template, View) framework, that's (arguably heavily) influenced by the Django syntax.

GAAARH!

Sometimes this project just feels like putting out fires. Do something in one part of the code, and half of the rest breaks. Yeah, bad planning, bad programming. I'm THIS close to getting the framework's core finished. I just want to get this over with.

I just found out that I've introduced an infinity loop, a memory-hogging loop at that, with referencing keys. I think I'm going to make the foreign keys evaluate lazily also. But not tonight. I'm laying down my hammer and stepping away. I just might actually apply to our university's codemonkey zoo.

Update: Just sent the application...

Quote of the Month

"For each elegant solution, there's one show stopper that prevents you from using it."

Remember that one, boys and girls. Today I was hinted of a function that could solve a lot of the problems I described yesterday, especially the one where I could determine which column comes from what table. Namely mysql_field_table(), which does exactly that. I was somewhat astonished that PostgreSQL had the same function, which unfortunately queries the database metadata about the names of the columns, but that's better than a foreach-loop in PHP. But the show stopper was SQLite's deficiency of said function or equivalent.

While SQLite is somewhat of a marginal database when it comes to popularity, I've grown to liking it under this project. It's respectably fast (the major surprise) and it's very widely available. Many website space providers charge extra of MySQL or PostgreSQL, but they most probably have SQLite support (unless they're cheap and wants to rain on your parade, just to get more money out of you). So that's why I want to have SQLite support included in this - the framework's greatest requirement is the support for SQL, and with SQLite, you get SQL support with virtually any PHP-installation.

While I'm at it, I can openly say that what I'm doing with Models and Fields is practically a variant of Propel. With some regards, it's not as robust and fancy as Propel, but on other areas it's more intuitive (definitions in native PHP code instead of XML *shudder*) and flexible (faux custom aggregate functions on a per row/model basis).

Now, back to the drawing board. I have stuff to think about.

Frightening Complexity

I'm officially working with spaghetti code right now, and it's all because of the referencing forms, and I'm thinking of stopping that before it gets all too entangled.

I generally dislike discussing code particulars on this blog, but I won't stop myself this time. Currently, I have a working ForeignKeyField, that is actually committed to the repository. The good thing is that accessing it with PHP makes the Model 'on the other side' completely transparent. Hugely simplified, doing "$myModel->foreignKey->foreignTableName" would do what your intuition says it should do. The bad thing is, until you evaluate $myModel->foreignKey, there's no way of accessing foreignTableName. The upshot of this all is that you can't select a $myModel based on the values in the foreign table. You can't select a person (or a collection of persons) based on the color of her(/their) car(s), if Person and Car are two separate models, with a foreign key between them.

So, let's, just for a moment, assume that this is not a big deal. Sure, it's not all that common to select persons based on car colors. Let's assume that it's completely reasonable to look through each and every person and respective car's color. The complexity is still O(n). While respectable, noticeably worse than O(1) that we would get if we could 'look through the foreign key'.

The deal gets quite a lot bigger when many-to-many relations come to play. Pizzas have many toppings, and toppings are on several different pizzas. So you have Pizza, you have Topping and you have a table joining the two, saying which topping is on which pizza. Doing, Big-O is through the roof, and we have a O(n²) on our hands, if we can't see through foreign keys. We are forced to do $pizza->foreignKey->foreignKey->toppingName.

The problem, however, isn't here. $a->fk->b is simple enough to fix with a join, and it's very easy to expand it to how many intermediate foreign keys as the user wants. I've actually done that already. One model can already be selected with the help of a foreign table's values. The problem is how the SQL requests are returned. As I'm currently only working on MySQL, it might be a problem only with that (and even if it's only MySQL with this behavior, I need to do additional RDBMS-unification code). But, I get no indication which result is from which table. This makes populating the result values in the correct models and their respectively correct fields. If I have a "name" column in the result, but three models, how am I supposed to know where to put it?

Anyhow. That's what I've been up to. How about yourself? Nice weather, I've heard.

*psst* helping hands are still welcome

Credits Where Due

I must tip my hat to the Django crew. The smartness of their Model code has wowed me from the first moment I tried out their framework.

I've got foreign keys working now. But they're not as fancy as with Django. You can't filter Entries with them, because the Entries object sees only the id numbers. And this is what's got me right now pondering about how to implement many-to-many relationships. Getting the integer value of the foreign key makes little value, as the search should 'see over to the other side' and 'through the join table'. I could simply not do them, and have the user do a model with two foreign keys. But that, again, is against my design mantra.

What really got me wondering with the Django implementation is that only one of the two models involved need a ManyToManyField, referencing to the other model. The other model that's referenced, but doesn't include a ManyToManyField still can access that join table, even if it's not mentioned anywhere. Without rummaging around their code, I'm thinking that either they have done something very clever, or then they do a lot of 'unnecessary' SQL queries, and I'm thinking it's the former.

Oh well. I think I've got some ideas up my sleeve. I'll see what I can do.

PS: I've put off doing the i18n for Fields. I'm sure it can be retrofitted transparently without any need of changes to user code. Getting the basics working is paramount at this moment. I might actually postpone the i18n even further to the future. I want the test version done and over with, so I could finally get some feedback.

Key Concerns

First off, Google's breaking apart. First Google Code was down for a couple of hours (a week-or-so ago), and now Blogger/Blogspot was down for a few hours.

But, back to the point:

I've done some tweaking with the code, correcting some typo-related bugs, and fixed some stuff previously done in plain stupid ways. This is a part of me postponing the start of any bigger feature. Just like I did miscellaneous smaller stuff before starting on the Model/Field pair, I fixed previous code before doing any foreign key -related logics, which will translate to ForeignKeyField and ManyToManyField.

Actually, I'm not sure if I like those names for the fields because they relate too much directly to relational theory, instead of being intuitive and reflect what they do/contain. Anyhow, the name is a minute detail, it's easy to change, as long as it's changed before wide-spread use.

But, I hit immediately to a snag. I haven't yet thought of a way to translate the id, stored inside the Model, to something that I can give to the Field, so that it can search for the matching referenced model. I would really hate to force the user to manually transfer the id to the field, which would go against the "maximum automation" mantra I adhere to.

Once I get the referencing fields working, I'm thinking LightFrame being ready for live testing... I could write user documentation while waiting for this to solve itself, but I'd rather not.

Update: Forget the above... Where do I need the model's key anyhow? Things are progressing just nicely.

We Have Commit!

There it is. The Commit.

This is the way to blow your SVN quota, shuffling files to and fro, shuffling function positions and such. Anyhow. There you have it. Models seem to be working. I'll detail the inner workings in the wiki in the following days, so if you are impatient, you are free to look at the code.

Right now, I need to get to bed "early", because I have an inhumanly early wakeup call (07:30). *yawn*. Night.

Commit Imminent

I must check the source once again, but I think I just finished Models... That obviously can't be, as... well... it just can't be. I must've missed something fundamental somewhere. But, for now, let's assume that Models are finished. Therefore my (one) celebratory cider wouldn't be unjustified.

However, there is no commit yet. I'll sleep on it, and take a new look tomorrow. If for no other reason, some code could be cleaned up. But I'll commit tomorrow, and, as previously hyped, it will be with wide-spanning consequences. Don't be lulled into false hope, though. This isn't the testing version I talked about earlier. It's just a commit where Models are working. Actually, Forms, for one, are far from complete. They are working just enough to give Models a go.

Hm, I guess I need to do a tester's guide before releasing a test (read: alpha-ish) version, so that people can get it up an running. *makes a mental note*

This is what you get from the amount of design and planning I've done. I have some guidelines, which you can read on the wiki, but nothing fine-grained. The lack of a clear documentation has bit me in the ass several times during Models alone, but it has been worth it. I can guarantee that if I had had done throughout documentation and planning from start to finish before I wrote a single line of code, I probably would've never got as far as writing a single line of code. Therefore I'm currently thinking ahead as much as I possibly can, but still more or less winging it. But, better a bunch of crap (yet working) code that needs refinement later on than a heap of shining plans of excellence (+1), but a lost momentum and motivation.

Anyhow. Cheers, and more stuff tomorrow.

Exhiliarating Babysteps

*plonk* goes the closed issue. How sweet a sound this is.

Yes, I closed another one of my 'issues'. I use quotes, as this isn't as much as an issues list, but a todo-list. But it's quite practical, and does that job quite well, I might add.

I really, really like the way things are progressing right now. I must say that while knocking on wood. The whole Model package (including Entries) is more than half-way done, and I see somewhat smooth sailing until the end of that. Without making any promises, and trying not to give Murphy too heavy a push, I might actually do a commit (a biggie) before the end of the week, that would include a working Model.

I see two possible showstoppers between now and when I'd like to declare LightFrame "ready for testing": The first one is foreign key fields and many-to-many relationships, and the second is built-in dynamic internationalization for Models. There is naturally a bunch of other things to do - such as writing more tags, more filters, finalizing SQL support and develop lf_setup.php - but I see those mentioned earlier as the major hurdles.

But. "Babysteps" is the word. Next up: query result ordering.

PS: 2669 lines or 65.4KB of commented code, according to my calculations.

More RDBMS incompatibilities!

Oh shi-

I just noticed that MySQL's string comparisons are by default case insensitive, while PostgreSQL has case sensitivity. MySQL's behavior can be changed by collations, while PostgreSQL doesn't support that. So, to make PostgreSQL case insensitive with CPU-intensive comparisons, or to make MySQL case sensitive, probably making a mess of the portability on the same go - that is the question. Oh, I haven't dared to check out SQLite yet.

From the quite limited experience I've had with the more specific differences between the two major SQL engines, I'm calling it a tie for now. PostgreSQL really tries to be a solid champion, I'd imagine it aspires to be a free version of Oracle, but some implementations might limp a bit. MySQL on the other hand wants to be the hip and pop engine, that does everything and is really proud of the spotlights its getting. It's actually so fascinated by itself that corner cutting is always an option, because it knows that everyone will love it nonetheless.

PS: I haven't done much actual coding tonight, due to the Wikipedia-effect I get when searching for stuff I need to take into consideration. Here, catch!

Musings About Exceptions and Errors

What's the semantic difference between triggering an error or throwing an exception. Seems like a clear cut difference, doesn't it?

But it really isn't all that crystal clear, I've noticed. Just search on Google with "exception vs error" and you get a lot of hits, and while many go on what marvelous things you can do with try-catch-finally blocks, not many (at least of those I looked into) go into great lengths of actually describing their difference. Some pages seem to want to answer the question, but lose themselves in the middle of thing, and veer of talking about something completely different. It seems like it isn't too clear for many programmers.

As a concrete example, LightFrame's database connection. You create a SQL object, which establishes a connection to the RDBMS you have defined in the settings. Let's assume that the connection fails. Should the object trigger an error, or should it throw an exception? If you prefer to catch an exception, what do you plan on doing with that exception? Correct your settings in run-time? I know you have an objection already, but bear with me...

LightFrame is currently set up with custom exception and error handlers. If an error occurs, or an exception is uncaught, a 500 HTTP response code is given, with an accompanying message and a backtrace of function/method calls. So, an uncaught exception is, for all practical purposes just a glorified error.

As I already saw in some of Google's results, there IS a difference between errors and exceptions in PHP that is non-trivial, and they should be treated that way. It is meaningful to use both, and overusing exceptions is something that shouldn't taken lightly. A good differentiation between errors and exceptions is that errors occurs when the shit has hit the fan, and you are pretty much screwed - die and get it over with - while exceptions are events where something has gone wrong, but there's something that could be done to remedy the situation, often, trying the same thing from a different approach.

To get back to the database example above, a connection error would do good to throw an exception. When the exception is caught, one could inform the database administrator about the problem via, say, email. In some cases, the data that was about to be gathered from the database isn't all that crucial and can be disregarded occasionally, or replaced with something generic. As a counter-example, an error should be preferred if there is actually something totally wrong. For example, if you try to use "excel" as a database backend, LightFrame will not try to fix that on run-time with an exception. It's a clear cut case: Excel isn't supported, your configs are wrong.

One must wonder whether these musings bite me in the ass someday...

The Train Is a-Rollin'

I don't know what I did just there, but I really have got back the momentum.

I'd say that the Model class per se is pretty complete. The stuff around is is not very much so, e.g. I only have three types of Fields, and the database interface is only 50% done. But Models should kick ass right about now.

The 50% I just mentioned is, by the way, somewhat literal. Unlike Django, that has its models always go through a Manager until you can access the database, LightFrame has a perhaps even more intuitive way of accessing the data. Data retrieval is split into two methods - single results, interfaced with the model object itself, and several results, interfaced with a special Entries object. The reason is that data is handled 50% of the time as single entries (showing individual blog entries), and 50% of the time as a collection of entries (a blog archive). This enables the developer to be more assured which quantity he/she is dealing with. LightFrame can guarantee that you only delete a maximum of one entry, when you delete it with a Model.

On a related note, the code is going a major reorganization. Files are thrown about and there's poking and tweaking everywhere. The next commit will contain a working Model system, and that isn't all that far in the future, I might add. Ooh, I feel somewhat giddy inside as a first test release is nearing fast. But the fixing rounds that ensue just after that aren't something I look forward to...

On an unrelated note, I haven't heard from Joona since he took a checkout of the code, so I've taken him off the contributor list.

Google Doesn't Play Well With Google

It seems like Googles own software is borking up...

I use Google Analytics to monitor traffic (what traffic?) on both this blog and the Google Code page. The Code-part gets a multiple amount of hits compared to this blog, according to Analytics. But, when I see a unknown source (i.e. something I personally haven't set up) linking to it, it's aways a site linking to either Google Code in general, or some entirely different project within Google Code.

Sure, I understand that it's a bit of a hack to provide Analytics codes for something like Google Code. But you'd think that those guys making over $100k a year would get it right. The weird thing is that I'm getting just isolated hits from these stray links, it's just single hit counts, like one or two per source.

Well, paraphrasing a would-be employer "you only need to finish a project 95%, the rest 5% is just inefficient polish" - well, I guess these stray hits are in the inefficient segment...