It’s been many months since my last post. My Perl API for PALO has seriously stagnated. What do I have to say for this and am I sorry?
Well I guess it’s a mixed bag I wrote a fair amout of perl and become frustrated after conducting a performance test against palo.
I actually lost interest in my API when I conducted some performance tests using the Palo HTTP API when creating and deleting elements in a dimension. I don’t remember the exact scenario, I think I was testing against Palo 2.0 and creating 10,000 elements in a dimension. From what I can recall the performance decreased in a exponential fashion the more elements were added / deleted from the dimension. Maybe this wasn’t a fair test, how often do you want to delete a large amount of elements.
So what has changed ? Well for a time I actually abandoned the perl API and lost interest in palo! Since then I’ve changed company and lots of water has passed under the bridge. Having a new place of work and a more friendly energizing working environment has given me a renewed sense of purpose and vigor.
Palo 2.5 has been released and I’ve spent the last few days writing a piece of java to allow the rapid prototyping of cubes. As a former consultant I frequently found situations where clients didn’t fully comprehend the principles of dimensionality. They didn’t necessarily understand hierarchies or didn’t actually know quite what data they want to be able to analyze. Generally after you had developed the ETL routines and they had used the system their level of knowledge would increase and they would soon come up with a whole host of change requests.
From my perspective this was frequently annoying, extra routines would need to be created and decisions made as to if the data for the missing levels should be back filled or defaulted to “not defined” type elements.
With this new tool I hope to be able to hit the ground running so to speak and be able to educate the end users from day one. The real beauty is that it will be possible to create a visual model representing their problem domain. They can freely change the model to add extra dimensions, levels, measures or cubes and then the tool will create the cube for them and populate it random sample data. This offers the ability to play with data and determine if this will meet requirements and also the fact it’s possible to do all of this in less than 5 minutes.
I hope to release the tool within the next week, it’s essentially complete but needs to have a few extra GUI components added allow you to chose which server and database the structures will be created on. Watch this space for more details. Adding the final elements shouldn’t be too much effort, creating a screen cast showing the tool in use will be much more effort. I’ve used a great free tool called WINK (http://www.debugmode.com/wink/) in the past for this. Wink can capture your screen, you can then add comments and render the whole thing to a flash file. Its great for creating demonstrations and tutorials. There are also versions for linux and windows. This time I’m doing the recording on the mac and have used another free tool called Jing. I haven’t figured out how to add captions, so you will have to listen to the audio instead. I’m sure this quick recording with give you an idea of the tool, yes it would be more impressive if I included slicing and dicing in excel or using the jpalo web client.
Check out the video here.
Some more astute readers may also be wondering what I’m going to do with the perl API. Well I intent to complete it and make it available to all. After using Eclipse for my java development I’m become a fan of this environment and have installed the perl EPIC plugin. This has proved to be a real revelation, I now have advanced features such as debugging using breakpoints and code complete. WOW Life seems so much easier now, no more text editors and print statements for me. I hope this will increase my productivity so I can finally finish this project. Will I ever use it in real life ? I don’t know I really like jpalo and think Tengegrity have done a great job on creating such an intuitive model also I don’t actually work with cubes any more and have focused much more on ETL. Buts it always good to have the option.
Watch this space
1 Response to “Rapid Cube Prototyping”