only this, and nothing more

Robots

2011-04-04T15:10:00.000-07:00

My significant other works in a bio research lab for the University of Washington. She sometimes uses a machine that she refers to as 'the robot'. I work at Amazon on AWS/S3. The closest thing I get to a robot is our soda dispensing machine. My office/desk needs a robot. hmmmm.....

Coffee shops

2011-02-27T15:06:00.000-08:00

I'll admit that I live in Seattle, a city lucky enough to have weathered the recession better than most. And yet, Seattle is the home of Microsoft. As such, I would expect to see mostly Windows laptops when I wander into coffee shops. Historically, this has been true. There were always a few trendy folk (or well of dorks, such as myself) with some for of Apple laptop, but they were the exception. The tide has been trending toward more and more Apple products though. Today, I was struck as I walked into my neighborhood coffee shop and counted 5 laptops, with not a single Windows machine in the bunch. Soon, there-after one laptop user left and was replaced by the lone Windows laptop in the shop.

Has Microsoft mind-share fallen this badly? I know that when I talk with co-workers, many of them have Macs at home (or Linux... I do manage a team of software developers after all). This is despite the fact that many of them are ex-Microsoft employees. Many of us use Windows at the office, but it is telling that given the choice, more and more people choose Mac. I realize that Seattle is its own little microcosm, but I fear that this does not bode well for Microsoft. How long before the average person's only 'computer' is an iPad or Android type device?

We are moving to a world where everything is on the web. You don't need a large hard-drive because you can archive everything into the cloud, or maybe store everything on a computer that sits in your closet, or (more likely) next to your TV. Near ubiquitous internet connectivity fundamentally changes the game. Web applications for all your core tasks fundamentally changes the game.

Microsoft is just like the other dinosaur industries mentioned in Innovator's Dilemma. The pillars upon which their business depends are eroding. I fear that soon they will discover that they will wake up and discover that even their precious enterprise market is no longer safe.

Time to start think about selling my remaining MSFT stock.

Where did he go?

2011-02-13T17:20:00.000-08:00

I have spent much of the weekend working on some browser-based tools to access S3 and manage content stored there. In talking to a few people about what I was working on, I often described it relative to a typical blog. Eventually that lead me to look at my own blog and realize that it has been over a year since I posted. Ouch.

So what have I been doing? Lots! Work (Amazon/S3) keeps me very busy. The last year has seen us continue to grow and launch feature after feature after feature. My team is hiring like crazy as well. This summer I spent 2 weeks in Maine with my family, trying to keep my nephew from riding off the roads into the bushes. I spent the spring climbing mountains taking the WAC's Basic Climbing Class. (Highly recommended by the way.) When not out hiking/climbing, my weekends are often occupied with walking/jogging the dog and generally enjoying Seattle.

I spent New Year's down in the SF Bay area visiting a friend, and talking to people down there made me realize how much I really love living in Seattle. It helps that so many of the tech people I talked to down there also use AWS. It is hard to imagine a better place to be.

Books, Audio-Books, Bray, Stross, and some new decade thoughts

2010-01-24T16:19:00.000-08:00

Tim Bray just wrote two entries that reference Charles Stross, far and away my favorite recent sci-fi author. He started with a reasonable review of "Saturn's Children", which is have slowly been working though as an audio-book. I've read a number of Stross's books in paperback, and happily introduced others to my addiction. Not since picking up Snow Crash, have I had such an experience with a new author. His stories are denser with contemplative ideas than many books on the non-fiction shelves. Saturn's children, to my mind, isn't his best, but that is heavily influenced by the fact that I am listening to it as an audio-book.

As audio-books go, the reading is incredibly good. I've listened to audio-books on long road-trips for years. Ever since since Nat and I read 'The Lion, the witch and the wardrobe' series to each other going to/from Burning Man, I've been an audio-book convert. The nature of the medium means that it is slower than reading. I don't have many moments that are opportune for an audio-book. I listen to pod-casts when walking the dog. Maybe if I had an iPod doc in the kitchen I'd listen more, but I usually have NPR on. I just haven't found a good habit/rhythm for audio-books.

Bray also discusses a recent essay from Mr Stross about the Google Books settlement. I haven't been following the Google Books settlement too carefully, assuming that old-media was just trying to hang on to their aging business model. Stross does a good job arguing why aspects of that business model are appropriate. He isn't arguing that things shouldn't change. He is asking how can we preserve some of the benefits of the old model. If information is free, then author's like Stross will have a damn hard time making enough money writing books to pay the bills.

I'm curious to see how this all transpires. I expect that how 'information' is priced and purchased, will continue to change dramatically. Very few things of real value are truly free. In the beginning, there were pay-walls. Then you have ads. Now they are collecting monetizable data on our behaviors. The latest evolution comes from services that make it cheap and valuable for individuals to generate content (ne twitter/gmail/search) while at the same time they are generating value by monitoring the content in aggregate. It is easy to see how smaller content (IM/email/blog-post) works in this formula. Large content, such as a short-story, or even a full-length book are more challenging. Giving it out for free doesn't compensate the author comparative to their effort. This seems similar to some of the arguemts I've been hearing about why Hulu will start charging for some shows. It is a simple value/reward problem. Do you value the content enough to justify spending your hard won cash for access? I know that I choose to buy content on regular basis, be it book, movie, or music.

The last 10 years has seen the rise of the iPod and iTunes along with their switch to DRM-free formats. It has seen the digital camera all but replace film for all but the most serious photography work, resulting in a proliferation of free content hosted online, dramatically changing the business models for stock photography. It has seen the rise of online access to video content, initially as a pay-to-download model, now in a mix of ad supported streaming and pay-per-view streaming. In 2000 everyone could self publish online, but few did. With the rise of blogging, then the social web, anyone with some internet access can claim their corner of the web, choosing to enlighten us all with their thoughts, artistry, or just entertain themselves and their friends.

What will the next 10 years bring

Lady GaGa vs Depeche Mode

2009-11-21T23:31:00.000-08:00

My pop fascination with Lady Gaga has now been justified by a remix of Lady Gaga's Paparazzi with Depeche Mode's Just Can't Get Enough. DJs From Mars, I thank you.

p.s. DJs From Mars MySpace page has links to more videos for similar remixes.

New Toy = NetBook EeePc 1005HA

2009-10-07T20:41:00.000-07:00

Just got a new toy for our upcoming travels. I wanted something small to dump pictures onto, so I can label them while I still remember the details. I used to have a Fujitsu P2000 series that I used for this, but that poor Transmeta chip wasn't really holding up very well. So I bought a Asus EeePC 1005HA. The first one arrived with a busted wireless. Sent that back and am writing this post on its replacement.

Aside from spending all last night installing all the necessary updates (See previous post) I am rather liking the little blue beast. The keyboard is small, but usable. The screen is also small but usable. Google's Chrome browser appears to be a particularly good fit, at least with just a few tabs open.

I'm curious to try it out more. I've got a 2GB DIMM to install, which should help things. I've also tried Moblin a tiny bit. I'll report back more when I've had more time to play with the machine. I'm really rather excited about a machine that has enough batter for me to play around most of the evening without running out of juice. I just wish someone shipped a 10" screen with more pixels. 600 pixels tall is pretty damn small, especially if you are used to a laptop 900 pixels tall. Everything appears freakishly big on the netbook at first.

If only it supported the two-finger up/down scroll like my MacBook... I miss that every time I use a Windows laptop....

Dangerous Data Structures

2009-10-07T20:15:00.000-07:00

We have been working on a particular component at work recently. The origins of this code go back to some of the very origins of the product. The code was written assuming a set of facts that no longer really holds true. My team has been patching it, trying to make it work with our new realities. We have come up with some pretty impressive hacks, but each fix has exposed new limitations of the code which we then need to try and patch.

What went wrong? Why was it so hard to fix the old code?

The problem was that the original code was using data-structures that didn't align with the new goals for the code. The core data-structure was of a rather neat design, but was also rather complicated. Both aspects deterred people from replacing it with something different. If the existing code was so complicated, the replacement (which must support more scenarios) must be more complicated, right?

I've seen this happen many times. Someone writes a neat bit of code, and later developers are nervous to replace it. Even though trying to retrofit new ideas on the old code is obviously painful, they would rather layer hack upon hack than rethink the original. This may not actually be true.

The worst cruft I have seen accumulate in 'legacy' code, happens when the original implementation used an inappropriate data-structure. Layer after layer of hack tries to pretend that the data is structured differently than it really is. This is one of those things that I think of when people talk about 'code smells'. This is one of the few times that a (partial) rewrite is in order.

It is critical that the data structures used are appropriate for the task at hand. Pick the wrong representation and now your code has to jump through hoops to do simple tasks. Pick the right data-structures and the code is clearer, and thus less buggy. With the right data-structure choice it is also easier to evolve the code, to add new features.

Just watch out for the day when those new features indicate that maybe you need to rethink your data-structures.

If Microsoft cared about Customer Experience

2009-10-06T20:15:00.000-07:00

I'm planning some upcoming travel, so I thought I'd try out a netbook. The first think I do on a new computer is download the latest patches. There is something seriously wrong with the fact that it is taking ~1hr to download and update a brand new computer. I realize that means my experience is about a fresh XP install, but since netbooks account for a significant percentage of laptops purchased today, I think my argument still holds. If so many people are buying these machines, doesn't Microsoft want a good customer experience? Instead, I get an insane list of updates. Why can't they ship a version of the OS that includes all these updates? That I need to waste hours updating, just so that I don't get a virus the first time I navigate to a random site with unscrupulous advertisers, is crazy. It just makes me want to try Linux. Have you seen the latest Moblin release? Apple understands this. When I bought a new MacBook latop last fall, I was up and running with the latest updates in minutes. The few updates I had to install were fast to download and quick to install.

Microsoft really doesn't seem to act like a company that cares about the customer experience. They care about the corporate experience... maybe. Amazon only just started delivering new laptops with Vista this summer! They are not the only company that kept away from Vista until they had no choice.

Microsoft is seriously at risk of loosing market share if they don't give customers a reason to want their products. I don't want Windows; I use it because it is the only OS with the apps I need. Meaning, Windows is the only platform with a good Exchange client, aka Outlook... If OSX really does get a good port of Outlook, I'd prefer that in a heart-beat. For home computing I use a Mac. I bought my parents a Mac. I think Microsoft is a lot closer to loosing a chunk of consumer market than they realize. I think Windows 7 is the last gasp before the consumer marked becomes fully commodetized. Soon after that the OS will stop really mattering.

Microsoft is dangerously like the steel-mills and mini-computer manufacturers profiled in Innovator's Dilemma, yet Balmer pretends that it is 1999. Sigh.

Wind Surfing

2009-07-20T10:35:00.000-07:00

For many years I've been interested in learning to Wind Surf, but never had anyone to go with me. The topic came up recently and Natascha showed some interest, so last weekend we took a Saturday afternoon class up at Green Lake. They have decent gear including some comfortably small sails, which make it much easier to start into the sport. Natascha and I both had a ton of fun. The weather around here has been perfect for learning; warm, sunny, with a light breeze. Yesterday we went back up and rented 2 boards for a few more hours of practice. I still spend more time in the water than I'd like, but am definitely getting better. Natascha picked a large sail than last week and got a serious work-out, but we both finished feeling like we are starting to get the hang of it. Now we just need to find some used gear so we can head out whenever we want!

Duckfeet in the snow

2008-12-18T10:16:00.000-08:00

I wish I had thought to bring a camera on my morning walk with the pup. Walking through Gasworks Park I noticed some odd prints in the snow. It took me a bit to realize I was looking at duck footprints.

Seattle Half-Marathon 2008

2008-11-30T20:29:00.000-08:00

Seeing Doug's post about running the Seattle half-marathon reminded me that I should mention that I also ran the half-marathon this year. Not nearly as good a time as my Vancouver half-marathon. I didn't train as hard, got sick 2 weeks before the race, and don't think I ate enough the day before. I managed ~1:50, despite having to walk a bit, for fear of passing out. Next year, I'll train for those damn hills better, and try and find a better pace person... Running the first 1/2 of the race at a 7:30 pace was not a good idea. Felt great at the time, but not sustainable.

Running 13 miles is an excellent excuse for some pints of Porter and a proper hamburger, and thus so did I indulge myself.

New toy

2008-11-30T20:07:00.000-08:00

Friday I picked up a new MacBook. (I almost bought a Samsung NC10 instead, but wanted a machine powerful enough to run Eclipse/etc.)

So far I have mostly been installing all my favorite Mac apps (Quicksilver, AquaEmacs, Fluid) plus a few developer tools (NetBeans, Eclipse, XCode) plus the usual (OpenOffice, Firefox). Given that my last Mac is an old PPC 12" MacBook Pro, this is a significant step forward. Everything is so snappy, and the bigger screen is a great. I've also moved over from my old Password Safe password archive to KeePassX and a DropBox account to share the password amongst my machines.

I've adjusted to the new trackpad surprisingly quickly. I love the new multi-finger scroll. One of the main uses of this machine will be browsing the web, and one of my biggest complaints about using my girlfriend's MacBook, is the lack of page-down key. The two-finger scroll is a great compromise.

I'm looking forward to getting some time to play with XCode an Objective-C. I started toying around with that on my old machine, but got distracted. Now that my household is mostly Mac (2 MacBooks + 1 Mini + 1 Work PC Laptop + 1 PC Desktop I never use), Mac tools are much more useful.

Been a long time

2008-11-30T20:01:00.000-08:00

I've been quite busy with work but unfortunately, the nature of what I do means that I can't really blog about it. Or at least, I have yet to figure out how to appropriately blog about it.

That said... it is the most interesting job I have had since when we were first working on XML support in Internet Explorer 5. (Back then I was helping define the XML DOM, XPath, and tons of other amazing things that people take for granted now.) Working on S3 has taught me more about the reality of building and running distributed systems, than I can even imagine fitting in one book. Working on hard problems with a great team, makes this one of my favorite jobs of all time. (p.s. we are hiring...)

Vancouver Half-Marathon

2008-05-04T21:42:00.001-07:00

Natascha and I spent the morning running around Vancouver, BC. 13 Miles of scenic vistas. Lovely day for a run. I'd forgotten how much I like Vancouver. The city is beautiful, and manages to have a bit more international feel than Seattle. The Half-Marathon was a very pleasant tour through a few parts of the city center. I'm not entirely sure what my time was, because we started late, but we think I managed just over an 8 minute mile. Better than I expected.

And I'm supposed to buy their content in the future?

2008-04-22T21:53:00.000-07:00

Yet again, Microsoft drops it's DRM on the floor. I avoid DRM whenever possible, but have used iTunes a few times for books on tape, and my significant other uses it occasionally for music and TV shows. One of the reasons iTunes is so popular is that it is as close to a constant in the industry as any other. Meanwhile Microsoft can't seem to figure out what it's story should be. This is why Microsoft needs Yahoo. Microsoft has no idea how to build a content business.

XML = 10 years?

2008-02-11T16:53:00.000-08:00

I saw the flurry of posts this weekend about how it has been 10 years since the initial XML recommendation was published. I particularly liked Tim’s XML People. It is amazing for me to imagine that it really has been 10 years. I still remember sitting in my office at TechnoTeacher (long gone) seeing the announcement about the formation of a W3C group to standardize on a subset of SGML fo the web. Way back when Usenet was still useful and I was a SGML newbie reading comp.text.sgml to try and understand some of the crazy stuff I was working on. There is no way anyone in that era could have imagined how XML would evolve and emerge as the ever-present beast that we all know and love. or love to hate.

XML arrived at an interesting time. The 'Internet' was exploding. Networked computers went from an office/academic luxury to an automatic assumption. What that meant was that many people needed to extend existing, incompatible systems to share data. XML appeared at a time when a flexible, international foundation for data exchange format was desperately needed. XML was not designed for this; it was designed for text markup! For all its flaws, XML came closer than anything else that existed. Big business jumped on XML like it was the philosopher's stone.

XML has been credited and blamed for many things. It has been used an abused. When the customer need appears and drives things that fast, it means many technologies/methodologies/etc get carried along for the ride. I'm not saying XML was not worthy, only that XML got dragged farther than many would have liked, and was stretched sometimes past what most deemed appropriate. XML was also one of the tools that enabled the explosion of commerce on the internet. XML changed the game. Before XML, people defined rigid schemas and binary encoded their data. Versioning was very painful. Writing parsers for data formats was infamously problematic and error-prone. XML arrived as processing speeds became fast enough to support text formats where binary formats used to be the rule. XML also solved a number of internationalization issues, by mandating Unicode and defining standard rules for determining which encoding was used. XML introduced the world to the concept of a 'self-describing' format. People who complain about XML today often fail to realize how much that they take for granted did not exist back then. No solution is perfect.

If XML really is as horrid as people say, then why didn't another format replace it? Because there was no other format that solved as many problems. It doesn't hurt to have some heavy hitters in your dugout, too. XML didn't become so omnipresent because of politics. It is here because it was a better 'good enough' than anything else out there. 10 years later and we are only just starting to see other formats, such as JSON, usurp the role, and only in cases where XML was not necessary a good fit to begin with. That is a spectacular success in my book.

A number of people have taking the 10 year anniversary to ponder 'what next?' To me it is obvious. XML is part of the plumbing of the modern web. It is not perfect. XML vs JSON debates amuse me. Each has its own place, and there are large areas where either works. XML should not be a religion. XML is a tool. I don't hit a nail using a drill, nor do I carve holes in wood using my hammer. We live in the world technology plenty. Just as a journeyman carpenter learns about his tools, developers should learn and understand the strengths and weaknesses of the tools at their disposal.

XML is here to stay. Use it wisely and prosper.

Remote Desktop for Apple Macs

2007-12-31T08:44:00.000-08:00

Last Christmas, I got my parents an iMac to replace their aged and infested PC. So far, this has worked quite well, but I keep running into the same problem: I need to walk them through some task, and doing it over the phone does not seem to work.

So what I'm looking for is remote control software that would let me see (and interact) with my parents machine over the internet. Ideally, I should be able to run the client from either a PC or Mac, so that I can walk them through something from home or at work. It looks like I have 3 choices:

Timbuktu - Expensive. It looks like you need a full license, just to run the client, which means I need 2 or 3 licenses, at ~$99 a pop.
Fog Creek's CoPilot - pay per use (5$), and I can't easily use it to log in and requires that my parents initiate.
Home-grown VNC solution - requires setting up a VNC server exposed though a ssh tunnel, as well as some sort of dynamic-dns so that I can find the machine.

Right now I thinking to use CoPilot to address the current problem, since that is cheap and easy. I've never used it though. Any caveats I should know about? Any alternate suggestions my Googling missed?

Back from Africa

2007-12-23T20:45:00.000-08:00

Just returned from 3 weeks in Kenya and Tanzania. Did you know that Tanzania is more properly pronounced something like tan-tay-nia? We spent some awesome time in a bungalow on the beach my Mombasa. We went to an amazing wedding in Nairobi (a friend of a friend was the groom). We spent 3 days on Safari in Maasai Mara. I'll upload pictures soon. Then we were off to Kilimanjaro. 6 days of sweating and I made it to the top! After that we were all exhausted and slummed it around the hotel mostly.

Overall it was an amazing trip. It was full of adventure and the unexpected. Hiking up Kili was the most demanding task I've ever attempted. I'm still amazed I made it to the top. The Safari was amazing. I loved seeing the animals, but it felt really odd driving a car around and chasing after these poor animals. Better than a zoo though.

And now after almost 24 hours of travel, we are home. My cat is curled up at my feet. And I have to work tomorrow with 11 hours of jet lag.

The many faces of a Cloud OS

2007-08-02T10:27:00.001-07:00

Dare posted an interesting rumination about the term "Cloud OS". This has also been on my mind recently, although my approach has been different. I am an engineer at heart, and have been wondering what it takes to build these things. I've been pondering Google world dominance, while I read up on building fancy Web 2.0 applications, possibly with something like Amazon's EC2/S3 services. Dare's post crystallized an idea that was floating, untethered, through my pondering.

This mythical Cloud OS, as with many myths, is multifaceted. The key facets that people are self-organizing around at the client platform and the server platform. This is really just an evolution of the traditional 2 tier enterprise architecture.

When talking about a user-facing Cloud OS, such as Jason Kottke's "GooOS, the Google Operating System", we are talking about a platform of Javascript and web-services. This is really just an evolution of the desktop app that talks to network file stores, and remote databases. Microsoft Windows has been replaced by a Browser, and the back-end services are much richer, but the abstract model is the same. This is what terrified Microsoft back in the erra of the original Browser Wars, and why they fought so hard to 'win' and then abandoned Internet Explorer development.

The flip side, the back-end platform that is needed to support this new client platform, is also sometimes referred to Cloud OS. From the client's perspective, this is actually the Cloud Database. This Cloud Database is not like any traditional database though, and itself requires a new underlying platform. Cloud Database has to scale in ways that traditional database systems can not. It needs to be built on scalable clusters of machines. It needs to be flexible and extensible. As mentioned in the interview with Paul Buchheit in Founders at Work, GMail has very different requirements from Google Search. This scalable platform of meta-services is also called Cloud OS.

Amazon's AWS services are a baby-step toward a platform for building the Cloud DB. Google's BigTable/GFS/MapReduce, as well as Hadoop/FaceBook (and others I'm unaware of), are the pre Cloud-DB. The single-machine OS has become a commodity. The evolution is now about this new server platform.

The client side is even more fragmented. There are new Ajax-y Javascript toolkits announced almost daily. There is no consistency between Google/Yahoo/Amazon/Microsoft/FaceBook client api/platfroms. The core data-models for such concepts are your contacts/friends list, email, spreadsheets, etc are different for every storage platform. This is data lock-in like we haven't had since the 80s. This side of Cloud OS is even less clearly understood than the Could DB.

We are in for some fun times folks.

W3C EXI Format Working Draft Published!

2007-07-18T12:54:00.001-07:00

The W3C just published the first working draft for the EXI Format. There is a lot still for them to do, but this is an amazing start.

Tales in (Good) Customer Service

2007-07-18T09:48:00.001-07:00

I recently had a friend tell me about the wonderful customer service he had received from Qwest after moving. This came as a surprise to him, as he was not expecting anything other than outsourced incomprehensible delay tactics. In his case, he got better support from Qwest than from Speakeasy. Just a few years back this would have been unthinkable.

I recently decided to upgrade my home DSL connection. I called and scheduled the upgrade yesterday afternoon. Everyone I talked with was pleasant and helpful. I did not feel rushed. They were good at explaining my options. When talking about TV package deals and my lack of TV usage, the rep on the phone related a personal story that was quite cute.

I finished the call with a plan for faster DSL service at a lower monthly cost. What do I remember? The personal story from the rep. I remember that he made the call friendly, unrushed, and inclusive. This is what customer service should be about. This is as close to old-fashioned small-town customer service as you can get.

Putting a pleasant approachable face on a company makes a significant difference. Qwest used to have a monopoly on phone service. Now with VOIP, I have many choices. If you have bad customer service, you are suggesting to your customers that they should look elsewhere. Customer Service is about keeping the customers that you have.

3 day, 1500 miles, the garden of eden

2007-07-03T10:05:00.000-07:00

Many many thanks to Ezra and Red Velvet for inviting me down. It was a long weekend, ~750 miles each way, but early summer in the NW US can be quite beautiful. Listened to "Stiff: The Curious Lives of Human Cadavers" on the way down, and "Life of PI" on the way back. Audiobooks are a divine gift for such a long drive.

The purpose of the trip was 2 days of camping out in a veritable garden of eden, with some amazing people and fantastic music... heaven on earth is dancing till your legs hurt, then hopping in the hot-tub, to mellow to the next DJ's grooves.

and I learned to almost hoola-hoop...

I needed a weekend without internet, iPhone, President Bush, traffic, and all the rest of the chaos. My world is now a better place.

Safari for Windows.. why?

2007-06-13T11:33:00.000-07:00

A lot of people seem to be pondering why Apple would port Safari to Windows. Safari is a minor player, so why bother? I can think of only one answer: iPhone and offline browser-apps

The iPhone has a Safari derived browser build it. Apple didn't talk about it, but if the browser is the development platform, then there must be something like Google Gears coming. If they want to get a large number of people building apps for the iPhone, they need to support development on Windows. The first step for that is Safari on Windows.

just my guess...

Useful tool: Synergy

2007-06-08T18:28:00.000-07:00

I saw a reference to Synergy on LifeHacker.com today. I know I've seen this before, but never really investigated. I now have it running on my work desktop (what better to do while I wait for tests to finish, and traffic to clear?). My normal setup is my laptop connected to a Dell E207WPF, and my workstation connected to 19" lcd next to it. I use a KVM switcher so that I only need one keyboard and mouse (and so the workstation can use the E207WPF). I do most of my work on my laptop, but I like to use my workstation for random tasks that I don't want to interrupt my flow, be they test runs, performance timing, or just iTunes podcasts. Switching back and forth is slow and annoying.

Synergy to the rescue. I now have my laptop set up as master and the workstation as slave. I can hop between machines with trivial ease. Well done.

Why use XML for serialization?

2007-06-06T14:11:00.000-07:00

In my recent post about WADL, I mention some of the problems with using XML as a data serialization format. In the comments, Eric asks why use XML for serialization at all? Browser-based apps are rapidly moving to JSON. Ruby developers prefer YAML. If XML really is such a mismatch for modern programming languages, why bother to use it?

I could argue about the availability of tools (XSLT/etc) to process XML. At AgileDelta, we often use E4X to help customers filter or reformulate their XML data. At Microsoft I often used XSLT, or MSXML + JavaScript to do quick-n-dirty processing of XML data. Really, JSON and YAML have a bit of an advantage here, because you can use the full power of your scripting language.

JSON is actually a hugely better than XML fit for many data serialization scenarios. It is more compact, faster to parse, and less ambiguous about how to map to your data-structures.

Where XML really has an advantage, is for formats which mix markup and data. Atom/RSS is a perfect example, as is XSLT. You could do ATOM is JSON, but then half your data is in one encoding (JSON) and the other half is in another (embeded HTML). With XML you get both, which is a huge boon when trying to process the data.

The other place that XML starts to have an advantage, is when you start having to support cross version compatibility. Eventually, you are going to have to write a loader that translates v1 data to your v2 data-structures. Browser-based apps can mostly avoid that. Enterprise apps can not. The closer you get to the database in a multi-tier app, the less that strategy works. The real benefit is that XML already gave you a serialization abstraction, so implementing data transformation becomes trivial. I'm not saying that this data transformation is not possible with JSON/etc, just more difficult.

Playing devil's advocate for a second here... given how few apps live long enough to actually need real back-compat layers, maybe the benefits of using JSON for V1 are worth it. Who cares if V2 is a days more work, if JSON shaved a week off your V1 schedule.

Back on point... XML helps by forcing a slightly higher level of abstraction. That also is realized by providing better facilities for validation of message contracts. As much as I dislike WADLs focus on XSD to the benefit of building RPC-like tools, XSD for message validation can be extremely useful. (Or use Relax-NG, or whatever other XML Schema language you happen to like) It provides a clear specification for what a message must look like. One of the down-sides of JSON/etc is that they tend to go unspecified. It takes a lot more work to ensure that your are not writing out properties that were not intended, for example. XSDs are not particularly useful in production, but they can be invaluable in development and testing.

So why use XML for serialization?
1) your data is a mix of marked-up text and structure
2) abstraction to simplify contract validation and cross-version compatibility

You could theoretically get most of (2) with JSON/YAML.. someday. XML provides that out-of-the-box today.

WADL: its really about the XML APIs

2007-06-05T16:55:00.000-07:00

There has been a bit of hubbub about WADL recently. Much of the conversation is about the need to describe the message using XSD and the benefits of WADL/WSDL for tooling. (I'd provide a Technorati search link, but Technorati appears down at the moment.) I think the real problem is that most XML APIs are so horrid to use. People want RPC (at least until they get burned enough to know better), and even when they know better they was the initial simplicity of RPC. One of the big problems with XML is that it is a horrid match with modern data structures.

You see, it is not that it isn't trivial to figure a way to serialize your data to XML; it is just that left to their own devices, everyone would end up doing it slightly differently. There is no one-true-serialization. So, eventually, you end up having to write code to build your data structures from the XML directly. The problem there is that virtually all XML APIs are horrible for this kind of code. They are all designed from the perspective of the XML perspective, not from the data serialization perspective.

It gets worse. XML is one of those things that looks really easy, but is actually full of nasty surprises that don't show up until either the week before you ship (or worse.., a few weeks after). Things like character encoding issues, XML Namespaces, XSD Wildcards. It is really hard for your average developer (who makes no pretenses at XML guru-hood) to write good XML serialization/hydration code. Everything is stacked against him: XML APIs, XML -Lang itself, XSD.

That is why so many developers (especially in the Java world) just use XML Binding layers.

Solution? Give them a good XML API. Not one designed for XML Gurus, who understand every nuance of the spec. Give them an API that makes using XML easy, and relatively efficient. This ain't easy, or it would already be there. XLinq is C#'s answer to exactly this issue. Java needs something similar. An XML API that isn't designed primarily for XML as text markup, but an XML API that is designed for data serialization.

XML Tools screen-casts

2007-06-04T13:42:00.000-07:00

Before leaving Microsoft, the last thing I worked on was the XML Editor in Visual Studio. In Visual Studio 2005, Microsoft shipped one of the best XML Editors out there. Intellisense, XSD syntax support, XSLT syntax support, XSLT Debugging, XSD Schema auto-generation. There is an incredible amount of functionality hidden in there.

One of my favorite 'hidden' features is goto-definition. Position the cursor on an element in a document with an associated schema, and F12 will jump you to the part of the schema that governs that element. I occasionally get some very complex customer schemas. F12 can also be used to navigate with-in a schema, from an element declaration to the declaration of it's type, or to the base type declaration. When the schema spans megabytes and multiple files, this is invaluable.

The XML Team just released some screen-casts of existing and future functionality. If you use Visual Studio and ever have to edit XML files. Watch Stan and you will almost certainly learn some new tricks.

Sasquatch 2007

2007-05-28T09:54:00.000-07:00

Back from Sasquatch. I only attended the first day, and my main motivation in being there was to see Bjork. Quick summary:

The Gorge is stunning. I'd never even imagined listening to such great music with a backdrop so fantastic. To my poor New Englander brain, it felt unreal.
Sarah Silverman was completely useless. IMHO, she shouldn't even get paid for her presence there Saturday.
Michael Showalter was entertaining and seemed much more comfortable with the MC role
Electrelane have potential... the live show left me wanting, but I could definitely see a good producer helping form their sound into a damn good pop album.
Viva Voce were intimate and beautiful on the side stage. Damn impressive to see the male half out there drumming while also playing bass to such lovely female vocals.
Citizen Cope and The Long Winters completely failed to impress. It wasn't that they were bad. I just like live shows that have edge and energy. They had neither.
MIA was well ... not there... apparently my friendly government found her form of music to be some sort of security risk.
Manu Chao rocked the house. This is what I want in a live show. Energy! Dance white boy, dance!
The Arcade Fire kept up the energy from Manu Chao, with their insane drummer, amazing vocals, and crazy musicians who seem to randomly swap instruments every other song.
Bjork was amazing. She performed some reworks of older songs that really blew me away. The laser show was simple and clean, in the best of ways.

Binary XML not quite so evil?

2007-05-24T17:50:00.000-07:00

I've recently been spending more time writing code that uses my company's Efficient XML (the basis for the forthcoming W3C Efficient XML Interchange format). I've never been one to claim the Binary XML will replace XML, rather the sweet spot where we target Efficient XML usage is places where the alternative is a custom binary protocol. I was reimplementing a tool that used Efficient XML to serialize some potentially large data structures. I was sure that I could do better and that using Efficient XML was overkill. (Yes... all programmers think they can do it better than the automagic tools.) So I sat down and implemented a custom binary format for the same information. I had the advantage of a working implementation that used Efficient XML, so the core implementation was pretty quick; a few hours. One of the goals for this is compactness of the result, and the data structures being written out had circular references and other things that ruled out any normal serialization tools I knew of. I did some quick tests on some really simple data, and it all looked good. Time to go home for the day.

The next morning, I picked it up and gave it some real data. Crash and burn. I spent the rest of the day tracking down bit alignment errors, and all sorts of small compatibility bugs between the reader and the writer. By the end of the day I had it working on most of our data. It still failed on a few cases, but they generated multi-megabyte output. I had no idea how to debug this. I added tracing, but the trace files were too large to load into memory! By this point I had already spent more idea on my custom format than it had taken to implement and debug the Efficient XML based code, and that code didn't have the benefit of existing, working code, when I wrote it!

AT this point it was at least good enough to evaluate the compactness of my custom format. My format was hand optimized, down to the bits. I expected to beat the Efficient XML encoded data hands-down. So I ran some tests. My custom encoding did beat Efficient XML for most samples... but not all of them! In fact, Efficient XML beat my hand coded format my 20% in one case! What was going on?!?

Well, I knew right away why Efficient XML was beating my code. I had skimped in one case to simplify the code. To achieve equivalent encoding, I was going to have to encode it the same way Efficient XML encodes such situations. The scenario is when you have a set of individually optional values. This is a pain to handle manually, and I have never seen a manual encoding that handles this optimally.

So what does this tell us? That Efficient XML can truely be as compact as a custom binary format! Since the format is specified by using XSD, there are a number of tools out there to help define and document the format. You can prototype the format in Text XML, and then switch to Efficient XML, once the bugs are mostly ironed out. Alternatively, you can manually decode the Efficient XML stream to Text XML for debugging purposes. I've found this invaluable.

When using Efficient XML (rather than a custom binary format) you are programming against standard XML APIs. There is a cleaner separation of the bit-encoding from the rest of the encoding/decoding logic. I have long argued that XML can be a good fit for configuration files, simply because it means there is less parser logic, and it is easier to user standard tools to process your config files. Much of the same benefits apply to Efficient XML, with the caveat that you need to either use APIs that understand Efficient XML, or translate from Efficient XML to Text XML. The import point is that you have all the options available with very little effort, all while getting many of the benefits of a custom binary format.

Efficient XML is no panacea. It is not a replacement for all binary formats, just as XML is not the be-all/end-all. Efficient XML is an excellent choice when XML would be a viable choice, except for it's verbosity. (Efficient XML is also faster than Text XML to generate and parse). I have also played with auto-generating custom parsers for Efficient XML with a specific grammar. These can be blazingly fast and yet still working with conformant Efficient XML.

Lots of people like to talk about why they think Binary XML is a bad idea: (a) (b) (c). Most arguments against Binary XML focus on 2 points:

Text is good, Binary is bad
XML is defined as a textual format. Anything else isn't XML.

(1) There are some good reasons to recommend Text formats. Any text editor can be used to edit the data. It is easier to debug. In packet traces and other debug logs, it is easier to extract and investigate. I definitely agree that Text is easier than Binary to debug and apply generic tools to. But! to compare Binary XML to a custom binary format is unfair. All you need is one conversion tool to convert any Binary XML to Text XML, and thus get all the benefits of text. In comparison, you would need a custom tool for every custom binary format. The extra effort means that the custom tool would likely never be written. With Binary XML, the tool is just a given. I have used this many times to great effect, for both Text XML and Efficient XML.

Summary: Text is better than Custom Binary, but Binary XML is more like Text than it is like like Custom Binary.

(2) The XML spec does define XML as a Unicode stream of characters, no-one can argue with that. But why then is it OK to talk about XML APIs? or the XML Infoset? When people talk about 'XML ' (or ' XML') they are talking about leveraging XML. In order to really do anything with XML, you need a parser, so unless you are writing an XML parser, you never deal with straight XML anyway. Binary XML just extends the existing XML domain to include a more compact encoding. You give up some of the benefits of a Text format, while gaining many of the benefits of a custom binary format.

Summary: Most software that 'uses' XML isn't interacting with the raw text stream, so why does this matter so much. Binary XML isn't XML, it is Binary XML.

Ultimately, Binary XML is not about replacing XML with some new binary encoding. It is about leveraging the many benefits of XML in situations that can not use Text XML. Binary XML just extends the reach of all those existing XML tools, both for the application developer and the application user.

plugging things in...

2007-05-17T09:44:00.000-07:00

This last weekend, I wandered down to the local Fry's to get me some gadgety goodness. Specifically, I wanted to pick up a Buffalo TeraStation and some wireless gear so that I could hide the TeraStation away. I've been trying to figure out how to run ethernet through the walls for as long as I've lived in the house, and have decided to give up. Wireless speeds are good enough now to make it not worth the bother.

I spent a long while in the networking aisle, trying to figure out the best options. My ideal setup was a 802.11n router and then something 802.11n to plug the TeraStation into. Thus my first dilemma. Every major manufacturer seems to be selling some kind of 802.11n router, and many of them sell USB/PCCard/PCI adaptors. My problem is that my idea location for my new TeraStation is neither near the DSL modem, nor near a computer I want to leave running 24/7. Thus I need either a 802.11n 'extender' or a 'game' adaptor, at least that seems to be what manufacturers are calling any kind of 802.11n client that has an ethernet port. Thus began my first problem with plugging things in... namely that I had nothing into which to plug my TeraStation!

It came down to 3 options:

(1) I already have 802.11g in the house, and there were a few 'extender's on the shelf. Annoyingly, and 802.11g 'extender' costs more than a cheap 802.11n router! I also was worried that a backup would saturate the network and leave me no bandwidth for checking the latest on Reddit! I could buy a router and extender, but I'd just want to replace them in a year with 802.11n.

(2) The old AirPorts could act as extenders, but I had been unclear from reading Apple materials whether the new ones could as well. They are also expensive!

(3) I've heard negative things about Powerline ethernet before, but the new versions promised up to 200Mb/s. It is comparatively cheap, and avoids saturating the house with more radio noise. I thought I'd give D-Link's Powerline HD Ethernet a try. It's all about the plugs baby!

I get home and unpack everything. I plug in one Powerline HD adaptor in my office, and another downstairs. I run their little setup tool and sha-za! They see each other! I was hoping the tool would also provide something that told how good the connection was, or some indication of what kind of throughput I might expect. No such luck though. The easiest things to do seemed to be to plug in the TeraStation, and time a file-copy. That required actually plugging in the TeraStation.

The outlet where I wanted to plug in the TeraStation was full, so I pulled a power-strip out of the closet and plugged the Powerline adaptor and the TeraStation into the power-strip. Everything boots up... but now the Powerline adaptor can't find it's brethren upstairs. I figure that the power-spike protection circuit in the power-strip are at fault, so I plug the Powerline adaptor directly into the wall... only to do so means unplugging something. Yea... I unplugged the power-strip with the freshly booted TeraStation attached. Thus began my woes.

I immediately realized what I'd done, and re-arraigned things so that I could plug the Powerline adaptor into the wall directly, and still power the TeraStation. The TeraStation booted up and began blinking at me with wild abandon. I figured it was just doing a filesystem check, so I went back to making sure that the Powerline adaptor was working. 15 minutes later, I can down to check on the TeraStation... it was still blinking and flashing like mad. The normal way to administer the device is via a browser/http interface, but the device didn't seem to be responding. The manual was less than helpful. Supposedly if you count the number of times the 'diag' led blinks over a 4 second period, you can figure out what kind of error it is attempting to report. I'm not sure if you have ever tried counting LED flashes while also counting the seconds... but let me tell you, it ain't easy! I decided to leave it be for a bit more. In an hour, I came back and it was still going! I know that it was trying to check 1TB of disk, but these disks were empty. I had yet to actually even use the device when I inadvertently unplugged it. At least the Powerline adaptor seemed to be working.

By the next morning, the TeraStation had settled down and I was able to connect to it and verify that all appeared to be working fine. Time for some bandwidth tests. First I verified that a trivial file copy worked. I was using Robocopy which copy speed in bytes/sec. All appeared good. Now copy over a ~1MB file. hmm.. I was only getting <1mb style="font-style: italic;">really slow. So I powered down the TeraStation and moved it up to where I could plug it directly into the router.. no Powerline ethernet involved. Now my laptop was getting ~12Mb/s! down to the basement, to get as far away from the wireless router as possible, and was still getting ~11Mb/s! Time to pack up the Powerline Adaptors, they are going home. No plugging for them!

The moral of the story? Powerline ethernet didn't work for me. oh yea.. and don't unplug a running computer.

** edit ** The product is called TeraStation... not TerraStation. Tera = 1,000,000. Terra = earth.

Email.. Thunderbird failes me again

2007-04-23T12:25:00.000-07:00

I saw all the news about Thunderbird's latest 2.0 release, and since I use Thunderbird for 2 of my accounts, I thought I'd give it a try. For my trivial usage, i.e. as a secondary email client, I see almost no functional improvement versus the previous releases. It has all these great features like GMail support, Tagging, and Search. It still doesn't address my biggest annoyance.

I use Outlook for my work email and GMail for the majority of my personal email. Thunderbird is just used for 2 legacy accounts that I have around for historical reasons. One is POP, the other is IMAP. Both are sssllloooww. The IMAP is via SSL. For both accounts, when I send email, I sit there with Thunderbird blocked on the send, for a few seconds. That happens for every message. Worse... because of SSL, the sends pop up a username/password dialog. But it takes seconds before it does so. By that point, my impatience has moved me on to some other task.

Here is what I'd love to see improved to make me rave about Thunderbird:
1) It knows it will need a username/password, and that it doesn't have one one file. Why did it wait?
2) The reason I don't let it 'save' my username/password is because it doesn't encrypt the saved username/password. If they added strong encryption of this, then that whole issue would go away.
3) It prompts me about expired SSL Certificates too often. Why can't I just click an 'always allow' check-box and get on with my life?
4) We live in a multi-threaded world. Why is send/receive still blocking? If Linux ran better on my laptop, I've be running Evolution, which I vastly prefer over Thunderbird. The main reason is that all mail server interaction is done in the background. I can delete/move/etc to my hearts desire and not have to sit and wait for every option to finish before I get interactivity back.

Don't get me wrong. Thunderbird 2.0 is better than the previous version. I am using it on a daily basis. For a free product, I feel guilty asking for more. Still ... resolving the above issues would make me a much happier customer.

It made me laugh out loud

2007-04-23T12:11:00.000-07:00

I love me a good comedy, but I'm not one to laugh out loud that often. (Which has lead many friends to question whether I'm enjoying that movie that left them rolling on the floor...) I don't know why I'm that way, but I am happy to say that given the right fodder, you will indeed hear me guffaw and see me approximate rolling-on-the-floor laughing.

What provoked such unusual antics? Hott Fuzz. Probably the funnies movie I've seen in a long... long while. A bit more gore than might be strictly necessary, but done with the same comedic skill as Sean of the Dead. I was also very pleased to note that unlike so many Hollywood action flicks, the Hero doesn't shoot to kill. The movie isn't all violence though. The dialog is excellent, there is actual story progress and character development (as compared to 300, which I watched last week). I'd have pre-ordered it from Amazon if I could.

warning: if you don't like British humour, B-Movie comedic gore, or have no sense of humour, you will probably not like the movie. If you liked "Sean of the Dead", go see Hot Fuzz ASAP.

Apple Delays Leopard

2007-04-17T22:21:00.000-07:00

It has been all over the net, and every one seems determined to speculate wildly as usual. On Apple's top news site they report that the next OS X release (code-named Leopard) will be delayed until October. They state that the reason is because "we had to borrow some key software engineering and QA resources from our Mac OS X team" to ship the iPhone on schedule. The net is speculating wildly about this, and it even deserved mention on This Week in Tech. A lot of people think that Apple is just using the iPhone as an excuse to delay Leopard.

I've never worked at Apple, but I worked at Microsoft for years. Apple's line makes perfect sense to me. On TWiT, they quipped that Apples explanation must be bogus because that would imply that apple is so small that moving a few people would impact the release so significantly. I think the TWiT folk are missing an honest understanding of how software development for products like Leopard and the iPhone really works. A few key developer can really have that much impact.

Jobs claimed that the iPhone was running a version of OS X, and that it includes a version of Safari. There have also been nasty rumors that the iPhone prototypes have horrid battery-life. That implies to me that they probably had to pull (A) kernel developers and (B) top/senior performance developers. Those are some of the rarest resources at a development company. Maybe 1 in 100... Maybe.

If they needed to pull testing resources, they probably needed to pull their top testing talent. The ones that have experience building test frameworks, and can handle a new, unfinished platform. Again, they are picking the cream of the crop.

Obviously, not everyone they pulled is upper echelon, but I be they pulled a lot of senior experience onto this project. That will have serious consequences for the teams that lost those people. 4 months delay isn't really that long in software development, especially when it includes August (or December) which is notorious for being when everyone takes their traditional family vacation.

Besides... The only Apple customers that really need Leopard are those lucky few with the new 8-core Mac Pro. The shipping version of OS X already puts Vista to shame. There is nothing in Leopard that motivates me to want it now. I can wait and laugh maniacally while I try and get Win XP and Vista to function near the level of elegance that my (now old) PowerBook had out-of-the-box.

May 15th 2007, RIP Internet Radio

2007-04-16T17:47:00.000-07:00

ArsTechnica has a great summary of the most recent ruling regarding the updated internet-radio royalty rules. I've been a big internet radio listener for years, and am sad to see this particular change. I'm sure the next target will be podcasts. I'm all for artists getting appropriate royalties, but I am sorry to see such an open forum killed by profits. The only salvation is the explosion of good podcasts.

Microsoft is Dead...

2007-04-08T21:35:00.000-07:00

Paul Graham's Microsoft is Dead essay seems to be creating quite the conversation. Don Dodge. Dave Winer. lots more. I'm amused that so many people do not get his meaning. Or maybe they just don't want to listen. Didn't they read Innovator's Dilemma?

Maybe I'm just biased by being ex-MSFT and still knowing a lot of people there. Here is how I read the tea-leaves: At Microsoft, 5 years ago people talked about leaving their team, now they talk about leaving the company. Virtually every individual I know and respect at MS talks about leaving. What was the last thing that Microsoft did that really mattered? Vista? me laughs. XBox 360? that is evolution, not revolution. Zune? maybe by V3. Office 2007? cool new toys...

Then there is the fact that I replaced my parents aging, zombie-ified WinME Dell box with an iMac and am helping my neighbor replace a dying Toshiba laptop with a new MacBook. In my humble opinion, most any non-techie who has the money should be buying a Mac, not a PC. User Interface matters, and Macs just work. The Web has finally become what Gates feared it would... Web+Html is the OS for most casual computer users.

Everyone is right... Microsoft is still a gorilla. But you know where the profits are coming from? Cannibalization. They are turning a team of amazing people into a lot of mediocrity. That isn't to say that there aren't great people at MS; just that there very few islands of real talent left.

Lack of elegance is viral

2007-04-08T19:13:00.000-07:00

I've been playing with PHP to build a simple web app. Amazingly, I've never really used PHP before. I have toyed with it (as well as many other many other web frameworks)but little more. Many years ago, I made my living writing Perl CGI scripts, but the state of the art has made some serious advances since then. PHP is not one of them.

PHP did make me realize just how spoiled I am. Programming in PHP makes me feel like I'm back in college, where I don't know the idioms and have half-baked libraries. The PHP language isn't really any worse than Perl. The PHP libraries are amazingly rich and I bet I could find something that could find the solution to world hunger somewhere in there. But that hints at my first problem with PHP. How the hell do you find the right library for your problem? It seems like there are 2 implementations of everything, probably because nobody could figure out to find the first implementation. There is absolutely no standardization across the APIs. The documentation is impressive in its quantity, but definitely not it's quality.

The biggest problem? All this ugliness encourages ugly programming. The best way to encourage good programming is a framework that is itself elegant. This is a large part of why Ruby (and Python) have such a following. I would argue that framework elegance is more important than anything else. PHP's mess encourages ugly programming because that the whole language/libraries collection is ugly. It is so hard to write clean code when every library call interrupts the flow.

This is one of the reasons for Lisp's cult following. You can mold the language to fit your purposes. Good code is about clean flow that makes the the logic of the code clear. When I look at the PHP code I tossed together, the presentation logic and the app logic are all a giant pile of ugly mush. While building and testing my trivial app, I would make stupid mistakes simply because I could barely read the code.

Ruby on Rails has gotten a lot of press as the future of web app development. I've seen startups advertise their use of Ruby On Rails as a hiring tool. A lot of people have weighed in on why Ruby and Ruby on Rails has garnered such attention. I think the reason is that the core language and the Rails framework are good examples of elegant design that encourage elegant application code.

TWiT and Moby's song "Go"

2007-03-27T12:25:00.000-07:00

I just listened to the latest This Week in Tech and near the end they are talking about the proposed Internet Radio Tax. Then mention Moby's song Go, the chorus of which has someone yelling "Go", and ponder how he might have acquired the rights to that sample. As a recovering Moby fan from back in his electronic heyday, I remember hearing an interview with Moby where he talks about the origins of that exact sample. It is actually a layering of multiple recordings of him yelling 'go'. So... contrary to the discussion on TWiT, Moby did not need to deal with sample royalties, as that is not actually a sample of anyone else's music.

This random, useless fact courtesy of my many many hours spent at WRUR and the continuing obsession with electronic music that developed out of those years.

Subversion: pro/con

2007-03-07T11:51:00.000-08:00

At work they used CVS before I started, but when I started working on my current project, I knew I would want better support for directory management. We mostly work in Eclipse, so I poked around for better alternatives. Subversion seems to have gained some significant momentum, so I thought I'd try it.

I live/work/breath/occasionally-eat off my laptop (Dell D610), and work from home/coffee-shops pretty regularly (gotta love startups!). SVN (aka Subversion) stores a local copy of your last check-out. This is great when working remotely. I don't need network access to review any changes I have in process currently. This has the downside of eating 2x as much disk, but disk is cheap. Overall, this is a definite win for my laptop.

I started using SVN right as the current controversy over possibly including SVN into Eclipse. There were 2 plug-ins implementing SVN support, each with different priorities: Subclipse and SubVersive. I won't go into the details, but waaayyy back then (aprox ~6 months ago), Subclipse seemed a bit less polished. It has issues with the fact that my Windows username is 'Derek' and my unix username is 'derek'. I prefer the Eclipse's CVS icons (as did the SubVersive team) over the TortoiseSVN icons. After a week of trying both, I settled on SubVersive and moved on to getting actual work done...

Then recently, I've been having all sorts of issues. I updated my Eclipse 3.2 install, and my first SVN action always failed, then the next try would give me a password dialog. The SVN Browser seemed to get particularly confused by this, and I would need to refresh the root. Then I couldn't check out a project at all. Fixing that by upgrading the SVN server somehow caused the SubVersive Commit code to get very confused.

So yesterday I disabled SubVersive and installed Subclipse. They still default to the ugly icons, but apparently you can change that (see Mark's comments on yesterday's rant). So far so good.

Overall, what are the pro/cons for SVN over CVS then?

Pros:

Directory versioning (file renames versioned, etc)
local baseline
atomic commits (not having this in CVS makes requires all sorts of hacks to do Continuous Integeration aka an automated build/test system)

Cons:

Eclipse/other-IDE integration not a solid as CVS (but it does exist and is much better than CLI or a toy weekend-project as for many other VCS)
Server data-store is not conducive to incremental backups
Still under very active development (= code churn = occasional bugs)
CLI support on Windows is iffy (getting svn+ssh to work is a major pain in the arse, and I hate how TortoiseSVN adds the icon overlays)

You can see that my Cons list is just a wee bit longer than my Pros list. The lack of atomic commits in CVS drives me up the wall, so having that fixed is a serious Pro. They really need to address the svn+ssh Win32 CLI support. Yes, I know you can make it work. I've done it, but every time it takes me >1hr. Telling users to run a real OS doesn't solve the issue. I'd be happy with a CLI implemented in Java if that is easier than figuring how to package SSH in a way that works out of the box.

I don't have a final conclusion, yet. There are many other version control systems out there, but CVS and SVN seem to have the best tool support. VCS integration into my IDE is one of my drugs of choise (almost up there with intellisense). After using the 'Team' features of Eclipse for CVS, I really don't ever want to go back. Other VCSs don't seem to have IDE integration even to the 2nd tier level that SVN has achieved, and that is a serious mark against using anything else. Given the progress of the SVN integration into Eclipse and the heavy use in public open-source hosting sites, I expect SVN to mature quickly. For now, I'm going to continue to use it until I find some reason to rethink.

Update:

Mark and I have been conversing on email, which pushed me to give the SVN CLI setup a 2nd attempt. I took a bit more time to look at the most recent machine where I got this working before and noted that you are supposed to double all the back-slashes in your subverions config file. For those following along at home open "\Documents and Settings\%USER%\Application Data\Subversion\config" file in a text editor. Search for the commented out SSH= line and replace it with something like mine:

ssh = c:\\tools\\plink.exe -2 -l derek -i c:\\MyDocs\\derek.ppk

As I am the only really user of this machine, I moved my 'My Documents' folder to the more convenient location "\MyDocs" which is then where I store my key for automatically logging into our SVN host.

This finally got my CLI working. Suggestions for the Subversion crew:

in verbose mode, log the actual SSH shell commands exec-ed (local and remote).
ship with a copy of plink and a simpler configuration setup.. for example, try logging in, via plink, if that doesn't work present a username/password prompt... Customers will be much happier with a awkward, but working initial setup and the ability to improve it as necessary, rather than the current most-likely-broken-on-win32 situation.

Eclipse Subversive (SVN) issues

2007-03-06T15:46:00.000-08:00

I spent 2 hours this morning trying to figure out why my Eclipse wouldn't allow me to check-out a project from SVN. I finally fixed it with an update the latest (1.4.3) SVN on our server.

Now Subversive fails with a network data error on every commit and doesn't think it actually commit any files. There is another Eclipse SVN plugin (Subclipse), but when I was trying things out 6+ months ago, subclipse was ugly and less stable. I guess it is time to try it again.

Right now, I'm about ready to give up on SVN entirely. CVS may have it's issues, but I really have no desire to waste time with craptastic tools. SVN's concept of branching may be clean in the abstract but is a pain to deal with. Why do I need to explicitly create the 'trunk' directory? I work with a small team, I don't need fancy tools. I need stable, working tools.

update: Subclipse works like a charm. Phew!

Bush's Budget

2007-02-22T09:24:00.000-08:00

Anyone who knows me knows I'm not exactly a Bush supporter, and never have been. I do find it interesting to watch most of the traditional Republican conservatives I know, turn away from the part due to Bush's leadership. I would love to be able to sit down with my intensely conservative, late step-grandfather and talk politics.

It came as no real surprise to read this article about Bush's Budget for 2008. Some tasty excerpts:

...a budget that offers $32.7 billion in tax cuts to the Wal-Mart family alone, while cutting $28 billion from Medicaid.

and

Cox family (Cox cable TV) receives $9.7 billion tax break while education would get $1.5 billion in cuts.

This is ridiculous!

I do wonder sometimes though. I think the journalist behind the article does the topic a disservice by opening it with a rant about Britney Spears coverage in the media. All that does is alienate people. Either your message is that Bush's budget is craptacular or that the main media outlets are letting missing the real issues. Mixing up the two messages helps no-one.

About usability...

2007-02-21T12:36:00.000-08:00

Via reddit I found Tantek posted about his "Three Hypotheses of Human Interface Design". An interesting read. I've always been interested in UI design, but tend to find low-level programming more interesting than UI development. His hypotheses definitely jibe with much of my experience. They especially remind me of my trial month with the Treo 700wx. I loved the fast web interface, and easy email integration. I hated the amount of extra work to navigate to common items, compared to my Treo 600 running Palm OS.

Another example, is the Quick Launch Toolbar. I use this extensively; every app that I use daily has it's a shortcut there. On my Mac, I use QuickSilver, and I've tried some of the similar launchers for Windows, but have yet to find anything nearly so intuitive. I tried to create a simple launcher myself, but discovered how hard a task it is to do well.

As a developer, I find it interesting to see how different development tools evaluate against this new measure. It made me realize that my main annoyances with Eclipse are all related to either too-many clicks/menus or slow/delayed UI reaction. For example... since most of my Run/Debug targets are command line tools, I often have 5 clicks + typing, just to debug something:

click: Debug [drop-down]
click: Debug ...
click: Configuration to debug
click: Arguments tab
type: edit arguments
click: Debug

A while back I learned I could control-click on the drop-down, so it becomes:

click: Debug [drop-down]
control-click: Configuration to debug
click: Arguments tab
type: edit arguments
click: Debug

But that only saved me one step. What I want is the command-line arguments on the 'Main' tab.

The 3 Hypotheses also explain why I find editing a project's properties so annoying. Lots of clicking, and sloooowwwww. Why should a simple dialog be slow on my damn fast development machine? I've never understood that.

The Hypotheses also mesh with Google's search UI ethic: Fast response, minimal effort, minimal noise. Interestingly, dropping 'fast', these are someof the design principles behind the new Office 'Ribbon' UI.

Worth pondering what sites you prefer and how that relates to these Hypotheses.

Everybody loves a Honda Civic...

2007-02-20T16:42:00.000-08:00

Everybody loves a Honda Civic, especially car thieves. Someone stole my girlfriend's Civic from right in front of my house last week, while we were sleeping. The neighborhood is so quiet, friendly, and family-oriented you would never expect such a thing to happen under your nose. I guess those 'club' wheel locks are a good idea.

Luckily, the police had already found her car by the time we reported it missing. Unluckily, it took Natascha an inordinate amount of time just to report it stolen. The problem was primarily that the police had a 'hold' on her car. It was sitting in a towing company's lot somewhere, but they wouldn't tell her where, and the towing company was even more reticent to reveal anything.

OK. Obviously as this is a stolen car, the owner would want their car back, don't you think? Apparently the police don't particularly care. They told her that they would call her when they released the hold. They didn't. They held the car for a full week, but they only pay for the storage fees for 3 days. They could hold the car as long as they like, and the car's owner gets to pay for the privileged of not having a car! Luckily, Natascha walks to work, but imagine if your attendance at you job depended on that vehicle?

The Seattle Police just went a long way to loosing that much more of my trust.

Restaurant Recommendation

2007-01-27T19:54:00.000-08:00

Last night, on a whim, Natascha and I ate at Ponti Seafood Grill. I have not eaten there in quite some time, but remember liking the food, and it is close to where I live. Either my memory failed me, or I just had an different experience last time. The experience I had last night was amazing! I had the pepper seared albacore, which was absolutely delicious (although too peppery for Natascha's tastes). Aside from the main course tasting so darn good, the overall pacing of the meal was excellent, and the staff was great. The sommelier had excellent recommendations, but more-over, I didn't get the normal, I-know-wine snobbery that I'm used to at upscale restaurants. The staff was very friendly and quite efficient. The final piece was their calamari... done right! The oddest bit of the evening, was that the owner was wandering around, chatting with tables. Often that would make me feel uncomfortable, but he really felt more like your neighbor.

Both Natascha and I had seafood, so I can't really speak to the rest of the menu. I can say with certainty, that I intend to visit more often.

[spelling update...]

For my birthday... someday

2007-01-19T10:49:00.000-08:00

Someday, for my birthday, I want one of these amazing sculptures. As a math dork, these are incredible. My desk wants one of them....

VisualStudio vs Eclipse

2007-01-18T10:03:00.000-08:00

I met up with a friend from Microsoft this last weekend and was challenged to explain why I prefer Eclipse over VisualStudio. The answer wasn't obvious. It isn't really that one has features that the other doesn't have. Both are slow memory hogs, and in fact, Eclipse is usually worse than VS on this front. Both provide good intellisense and refactoring.

So why do I prefer Eclipse?

In the course of talking about it, I came up with my answer. Eclipse if built by top developers for themselves. VisualStudio is built by top developers for some other developers. Both IDEs were built by teams of some very smart developers, but Eclipse developers seem more focused on building a product that they themselves would want to use. Microsoft has a strong internal culture that says that their primary target developer isn't as good a developer as most Microsoft developers. Which do you think will result in a 'better' product?

There are truths to Microsoft's delema. A huge majority of their target customers are not code gurus who think in boolean logic and can read hexadecimal upside-down. The majority of VS's customers are still VB developers who are more interested in drag'n'drop forms layout and easy database integration. I think Microsoft got it backwards though. They should be building a product that they love to use first, and then worry about 'dumbing it down'. Why? Because I think it will surprise them how similar these app-devs are to themselves. I think they would find that rather than 'dumbing it down', the would find that the app-devs have some excellent ideas for improving work-flow for common operations, that benefit everyone.

As an aside, this also explains why I liked Visual C++ 6 and put off 'upgrading' as long as possible. VisualStudio 7 was when they merged VB and VisualInterDev with VC++. Before that, VC++ was built targeting their own use-cases. With the grand union of all dev products, it was a compromise for everyone. I know as a VC++ user, I felt like I lost a lot, and I bet VB users felt similarly.

Minor Upgrade

2007-01-12T16:11:00.000-08:00

I just move to the 'New' Blogger. I'm not using any of it's features yet, but I hope to start using the tagging. If you use a aggregator, you notice the change because all my posts show up as new again...

MacFUSE make'n me jealous!

2007-01-12T14:39:00.000-08:00

I've been watching the FUSE project for a while, and am sincerely jealous. For a variety of reasons (mostly hardware and software compatibility) I spend most of my time in Windows. I keep holding out the hope that someone will implement something like FUSE for Windows, or even better re-implement FUSE on Windows, but no.

And now Google comes along and releases MacFUSE!!! Arrggg!

W3C announces EXI standard to be based on Efficient XML

2007-01-08T11:28:00.000-08:00

The W3C Efficient Xml Interchange Working Group just updated their published status to reflect that the group voted to base the forthcoming standard on our Efficient XML! This is big news for my company! I have learned a great deal, and my opinions have changed significantly, of the course of my working with different groups building 'binary' XML solutions.

'Binary' XML is a complicated and controversial beast. It needs to balance the desires for read/write performance and wire data-size. I have been involved in multiple 'binary' XML efforts, and am exciting to see the W3C working to build a standard.

There is a lot of confusion about 'binary' XML. In my mind, 'binary' XML is all about enabling XML for scenarios that were not viable before. The main problem that Efficient XML solves, is how to leverage XML over low-bandwidth links. Too many people are focused on using XML to improve XML read/write performance. While XML can provide some gains there, improving performance will not significantly enable XML where it was not usable before, but improves existing scenarios. By focusing on the limited-bandwidth scenarios, Efficient XML is already enabling existing XML services to integrate with clients that did not have the bandwidth to support those services as raw XML. Computing processing power is still improving dramatically, but bandwidth growth has been much more limited.

None of this is to say that read/write performance is not important as well. I've always been an evangelist for efficient implementations and the last year has been exciting for me. I am amazed at how much performance can be achieved while still prioritizing wire efficiency. I have been repeatedly surprised by how fast we were able to make out Efficient XML implementation. I am very proud of our work and have no doubt that it blows the socks off any other 'binary' XML implementation with which I have been involved.

I look forward to seeing further progress from the EXI group. As happened with XML, I expect a standard 'binary' XML will enable entirely new classes of applications to leverage XML.

Music: The Knife

2006-12-28T17:36:00.000-08:00

I recently discovered The Knife. I'd heard them on the radio, but then I saw a post of someone's best 100 (50?) music-videos of 2006 on YouTube. I can't find the link anymore, but The Knife showed up more than once. The more I listened the more I liked their sound. So yesterday, I picked up their latest, and I'm loving it. Very highly recommended.

For the record, my favorite music purchases for Nov/Dec are:

The Knife - Silent Shout
Ladytron - Witching Hour

I've been getting the music buzz a bit more recently and have picked up a number of albums, but those 2 are the ones that really stuck in my head.

Five Things People Don't Know About Me

2006-12-21T14:22:00.000-08:00

Arg... I've been tagged by Dare.

I sang in a Episcopal church choir for much of my childhood. One of my favorite aspects of going back to visit my folks for the holidays is to attend the Christmas Eve service, just to hear the music.
I played Violin for about 6 months in 5th grade. My fingers couldn't take it. I feel the same way now when I attempt to play Guitar Hero (and get schooled by my dear Natascha).
I'm supposed to wear classes. I was diagnosed as having Astigmatism in my left eye while in high school. There was a mandatory eye check, and I passed the normal check, then they made me try again with some glasses on. I passed that too. They then explained that I wasn't supposed to be able to read while wearing the glasses and that I should see an optometrist. The optometrist was quite surprised that I was an avid reader and enjoyed small-scale pen/ink drawing, since both should give me headaches. I wore glasses for reading through college and my first job, but after I scratched them after moving to Seattle, I just never replaced them.
I have never been arrested, but I did get a ride in the back of a police car. As far as I have ever been able to tell, I was picked up because I was a long-haired freak with dyed hair, running in a black trench-coat. Why was I running? Because it was horridly cold and I had just worked a 20hr day setting up and striking a TMBG show at my University.
I was one class shy of a minor in Cognitive Psychology. I generally found those classes more interesting than the Math and Computer Science classes that made up my major (Applied Mathematics). It wasn't just because the Psychology classes actually had women in them, but that did certainly help. <grin> The only reason I didn't take that one last class was because I made the decision to graduate early at the last minute and just didn't have time for that one class.

I generally don't like these things, so I hereby doom myself forever by breaking the chain.

Microsoft Search Brand

2006-12-01T09:31:00.000-08:00

First Dare and now Greg comment on MSN Search / Live Search. I agree with both of them, but I've also seen ample evidence that Microsoft's search still isn't as 'good' as Google. I do agree that Microsoft has a 'branding' problem. What is MSN? What is Live? Why do products keep changing names? Is Office XP newer or older than Office 2003? Lack of consistency trades off short-term hype for long term customer confusion.

This reminds me of a few years back when there was a ".Net" release of various products. These products had nothing to do with the .Net Frameworks, nor Visual Studio .Net, so it was rather confusing. What did the ".Net" in the product title mean?

Overall, I think these are symptoms of one of Microsoft's bigger problems. Marketing directs the product direction too much. This is a natural fallout of the growth of the company and the distancing of the development teams from the customer. The result of that distance is that the only people dealing with the customer is product support (which is often wildly undervalued at MS) and Marketing. In the end, you get strategy designed around buzzwords, rather than real content. You end up with ads with ridiculous dinosaurs in them.

Google jobs adds everywhere...

2006-11-28T12:28:00.000-08:00

It is the end of an era.

In the old days, you had to know someone on the inside to get a Google interview. Then the loosened up a bit... you just had to solve some difficult(ish) math/logic problem.

Now I see Google Seattle job adds on Download.com.

XQuery... worse late than never

2006-11-22T18:09:00.000-08:00

I just noticed, XSLT 2.0, XML Query and XPath 2.0 Are Proposed Recommendations. Since leaving MS, I've stopped tracking these. It baffles me that XQuery is only just now 'shipping'. This + the-mess-that-is-XSD spell the doom of XML. This isn't just doom, like the Outlook spell-checker would try an correct the DOM to; this is real doom. These standards to too complicated and too late.

I don't deny that big companies (and some very determined individuals) will implement them and people will use them. Lots of people will likely be putting XQuery on their resume in 5 years. What that will actually mean is that they used the 20% of the spec that SQL-Server/Oracle/DB2 implement. It is like SQL but so very much worse. SQL is complicated and isn't really standardized across RDBMS engines, but you can still learn the basics in an hour and be writing basic queries right away. I worry that XQuery is so complicated that it takes days to comprehend. That means that you may be able to write a basic query in 1hr, but there will be all sorts of edge cases you aren't aware of could actually impact your query.

I honestly think we would be better without XQuery. Let the vendors think for themselves and see what customers actually use. XQuery is a standard looking for a use, which is backward and guaranteed to produce a problematic result.

XSLT/XPath 2.0 is a harder one... There are a couple things that XSLT 2.0 adds that were desperately needed vs XSLT 1.0. But I've managed a team implementing a commercial quality XSLT 1.0 implementation and that was a huge amount of work. XSLT 2.0 is at least 4x as much work. That is terrifying. Why not just 'fix' XSLT 1.0? It would be dramatically less work, and provide 80% of the gains, at 10% the cost.

The type support that XPath 2.0 and XQuery provide carries a huge cost. Javascript/Perl/Python/Php/Ruby have demonstrated that you don't need types. Maybe XPath 1.0 had issues with type support, but XML and type-lessness are such a natural fit. I've played around a bit with E4X and find it very natural. Microsoft's X-Linq is also about leaving XML type-less.

Types in XML is really about mapping XML to traditional relational models. This should be solved by either improving the primitives provided by the database, or just having simple defaults for the obvious cases, and making everything else require explicit conversion.

I've found that I am using XSLT, less and less. Depending on my needs, I either use DOM + xml-writer, or I'm using E4X. XSLT is definitely the choice for document-y transforms (the priority rules and apply-templates is perfect for this), but I find it annoying to author (almost as bad as XSD). and anything more complicated than a page or so is write-once code.

Back to the XQuery pummeling... At this point, I think XQuery is actually a bad thing. It is complicated enough that most open source implementations will be bad or slow, which means that Saxon will remain the only real implementation. This means that people will continue to struggle with awkward APIs (the DOM) and in the end find their experience with XML less than appealing. I think that a simpler query language, that maps better to existing common languages (java/ruby/etc) and is simpler to implement would enable a much large class of user to benefit from XML.

Walla Walla Wining

2006-11-20T13:18:00.000-08:00

I spent the weekend in Walla Walla Washington. I've lived in Seattle for 10 years, and can count the number of times I've been to Eastern Washington with just one hand. Having grown up in New England, driving in Eastern Washington feels rather alien to me. I'm just not used to such large open spaces, without trees!

Natascha and I went out there to do some wine-tasting. We stayed at the Weinhard Hotel which isn't strictly in Walla Walla, but was close enough. We hit about 4 wineries Friday and again Saturday. I've now got 2 cases of good wines sitting in my basement. Now the hard part... letting the best of them age.

Overall a good trip. Highly recommended, especially in the off season.

GMail j2me client

2006-11-12T11:47:00.000-08:00

Brian asked if was able to get the GMail mobile (J2ME) client working on my E900.

I have successfully installed it, but I am unable to enter my password (because my password uses a symbol that I had been unable to enter to figure out how to enter into the password field). I spent all of yesterday at Mind Camp so I didn't get a chance to change my password. I just finished changing my password to something that can be typed on my phone and voila! It works! I now have gmail access on my phone.

Podcasts bloat

2006-11-09T11:26:00.000-08:00

I recently gave up on trying to get decent cell-phone reception with my Palm Treo and Sprint and switched to T-Mobile. I specifically wanted a GSM carrier, which means T-Moble or Cigular, here in Seattle. T-Mobile has better rates, especially for data (although it's data rates are much slower...). I also needed a new phone and copied my lovely girlfriend and got an unlocked Samsung slider. She got the D600 (which she loves), but I picked up the E900. It is smaller ans just looks slick.

The E900 has a built-in mp3 player and the J2ME implementation has support for MP3 playback, so I've been playing with trying to build a RSS/Podcast reader that I could use to listen to podcasts while driving to/from work, or reading the latest news tidbits in the Starbucks line. I specifically want it download the content and work offline. I've got some really hacky bits that accomplish some of this but and really feeling the limitations of the J2ME platform.

This has also caused me to go look at the various Podcasts I listen to and review file-size, etc. I was really surprised to see how many podcasters encode their dialog podcasts at 128kbps or greater! That is just ridiculous! I can understand why my Trance Tuesday podcast is at 192kbps (I am a music jukie, after all) but I don't understand why the Gillmore Gang is encoded at 128 kbps. (I don't mean to call out Steve Gillmore as the only perpetrator, but his podcast is near the top of my iTunes list...) Many of my other podcasts are at 56kpbs or 64 kpbs, which is perfectly fine for human speech. It also means much smaller files, so faster downloads, faster copies to you mp3 listening device of choice, etc. People.. please! If you are making a talk-show type podcast, choose an appropriate bitrate!

Security and Software Development

2006-11-09T10:30:00.000-08:00

I just caught wind of Jeff Jones' post about SQL Server 2005 [via OS News]. Apparently, in the year since it was released, there has not been a single security vulnerability! This is an amazing accomplishment. SQLServer 2000 was the source of the vulnerability exploited by the famous Slammer virus.

The last 6 years or so that I was at Microsoft, my team reported into the SQL Server org. (If you want to hear someone get riled up, ask most any XML-team member why their team is part of SQL Server after a drink or two.) I was involved with the security reviews and improved development process that went into ensuring that SQL Server would be rock solid and as secure as they could make it. The SQL Server org isn't the only place this is happening either. Similar practices are being used in Office and Windows. I'm looking forward to seeing how Vista and the new Office fare, security wise.

If I were at Oracle in management, I'd be looking at the graphs Jeff shows and at the customer erosion to SQL Server and MySql and pondering some serious rethinking of your strategy.

Rumsfeld out!

2006-11-08T11:03:00.000-08:00

Rumsfeld quitting; will be replaced by former CIA head

WASHINGTON - Defense Secretary Donald Rumsfeld intends to resign after six stormy years, President Bush said today.

This may be the best news I've seen today. I have very little faith in Rumsfeld's ability to lead out nation to any kind of 'success' in Iraq. I just pray that Robert Gates, the designated replacement, is strong enough to stand up to the politicalmaelstromm that is likely to result from yesterday's elections.

Java/Xerces 0, .Net/System.Xml 2

2006-11-07T16:43:00.000-08:00

I've spent much of this afternoon trying to figure out how to use an XSD schema to get a pscvi-augmented SAX event stream. Doing this is .Net with System.Xml is _easy_. There are samples in MSDN. Writing this code would have taken me less 30 minutes.

Now I realize that I am lot more familiar with System.Xml and Xerces' schema APIs, but this should not be a hair-pulling experience! I've been working on this for hours and still have no idea how this is supposed to work. The only sample I've seen (xjparse) has to peek into the XSD documents to figure out if the target-namespace is empty or not. This if ridiculous! I'm amazed any Java people use XSD at all! If the libraries make it this hard, I'd be tempted not to touch it either.

Yet againt, .Net's XML apis are vastly better than Java. System.Xml.XmlReader is night-and-day easier to use than SAX and then there is this. Oh right... Java's DOM doesn't include the Microsoft specific helper methods 'selectSingleNode' and 'selectNodes'... so maybe I should make that Java:0, .Net 3.

I'm going to go back to pulling my hair out tying to figure out how to use a specified schema file to add PSVI annotations to SAX events (via the PSVIProvider interface).

I appear to have figured out the API incantations. I ended up downloading the latest sources and stepping through the code to figure out what was failing and searching around to figure how to make it do what I wanted. My working code looks like this:


  SymbolTable symbolTable = new SymbolTable();
  XMLGrammarPoolImpl grammarPool = new XMLGrammarPoolImpl();

  if (xsdPath != null)
  {
    XMLSchemaLoader grammarLoader = new XMLSchemaLoader(symbolTable);
    XMLInputSource xsdSrc = new XMLInputSource(xsdPath, null, null);
    xsdSrc.setByteStream(new FileInputStream(xsdPath));
    Grammar grammar = grammarLoader.loadGrammar(xsdSrc);
    grammarPool.putGrammar(grammar);
  }

  SAXParser parser = new SAXParser(symbolTable, grammarPool);
  parser.setFeature("http://apache.org/xml/features/validation/schema", true);
  parser.setContentHandler(new MyHandler((PSVIProvider)parser));
  parser.parse(systemId);

It's all about the Mobile

2006-11-03T13:18:00.000-08:00

Opera releases a new Beta for their J2ME browser. Google releases a J2ME GMail client. It doesn't seem like that long ago that everyone was talking about how bad J2ME was... fragmented, awkward, slow.

I've got T-Mobile, so I'm not currently too worried about bandwidth costs, but I'm currious how bit-hungry these apps really are. This is definitely a space that I'm watching closely.

The real question: How are they going to make money off this? GMail Mobile doesn't include adds. Opera Mini is free and also has no adds... With GMail, the tie-in benefit is clear. Google is doing the polite form of embrace-and-extend. Microsoft has done this for years, providing lots of extras if you use their integrated services, but Microsoft is also famous for making it almost impossible to escape their embrace. With GMail, I use POP to backup all my GMail mails, so the only really issue is the email address.

Just as it tooke years for Google to emerge with such a winning revenue strategy for search (and online email, for that matter), I think we will see a great deal of gyration around revenue models for Mobile services.

The OS is almost a commodity

2006-10-30T22:10:00.000-08:00

I just got back from the pub, where we managed to drive one poor friend to pull out her Economist just to break up the computer-geek talk, but one of the points of discussion resonates well with Scott Grannemann's article "Surprises inside Microsoft Vista's EULA". Microsoft really is trying painfully hard to prove that it is the new 90's IBM. (You know... before IBM figured out Linux, Java, etc...)

My take on all this is that it proves that the Lawyers and Marketing folk run the show now. I can't see any other reason. The company can't make a profit by making better products, so it will squeeze every last ounce out of the customers it has, there-by making them even more likely to just ship. Even when it does have better products (MS Office vs Open Office, C# vs Java) it seems they are determined to dig their own graves.

The operating system is becoming a commodity. Look at Google, or at how anyone between the ages of 10 and 30 uses a computer. Microsoft needs to stop leaning on Windows and get back to competing! Look at iTunes. If Microsoft could ever build a decent mp3/audio app, they would then kill it by making it Windows specific. Part of the success of both MS Word and Excel depended on their Mac ports (at least from my experience). I know that the world runs Windows on their desktops, but they don't run Windows on their phone, or their iPod, or their Tivo. As I have said before, social computing is based on things like the iPod and YouTube. If Microsoft keeps limiting it's plays with Windows tie-downs (kinda-like tie-ins...) it will continue to fail. I pray that J Allard and co get this with the Zune.

Every day, Linux gets closer to being a viable daily OS for me (again... it has been years, but I lived by a Linux laptop 10 years ago, hard to believe). I've be writing this on an Apple laptop, except for my lack of trust of any V1 hardware platform. Windows no longer implies the tie-in that it once did.

My long road there: Binary XML

2006-10-26T12:11:00.000-07:00

First an admission: I've been 'doing XML' for a long time, and was not an easy convert to 'Binary-XML'. XML has many uses, and a curious history that is often at odds with it's actual usage. XML serves many masters. I've been using XML for markup, configuration, and quick'n'dirty data-stores for longer than I like to admit. For most of those purposes, Binary XML really just isn't necessary. In fact, I still don't think Binary XML is particularly advantageous. The advantages of being able to open the data in you text editor of choice far outweighs any of the benefits of a Binary XML encoding.

Yet... I have come to understand that Binary XML is a necessity. While I was still at Microsoft I was tangentially involved with SQL Server's XML support. One of the things I remember clearly was the motivation for a Binary-XML serialization: float to string conversion. Conversion from a IEEE float/double to an accurate text representation is horrendously expensive. While there are queries where you need to do this in the server, there is a huge advantage to offloading that to the client.

Since coming to work at AgileDelta I've been encountering another use-case that didn't really come up when I was at Microsoft: Constrained Bandwidth. At the extreme, when all you have got is a 300 baud modem, XML's verbosity becomes a real problem. This issue arises even in more common situations, very chatty web services for example. A few quick web searches show that a number of people are struggling with the issue.

It is worth stopping to review why people want to use XML. It isn't just because it is the cool thing to do. XML has an array of tools and frameworks that make it easier to use XML than building your own custom language or custom format. One of the big motivations to use XML is that you can depend on all these pre-built, pre-tested components to keep your focus on your primary task. Just as using Java/C# makes development faster/cheaper because the developers don't need to worry about memory allocation woes, XML provides faster/easier custom data formats.

What happens when you want all that faster/easier-ness but some part of your pipeline is bandwidth constrained? Or maybe you are struggling with a bottleneck such as the one I mentioned about SQL Server, where a significant amount of your precious CPU is being lost to XML serialization? This is where Binary-XML can save your bacon. I've watched some of our current customers implement solutions that leverage our Efficient XML to integrate limited bandwidth systems into an existing system with minimum of work, where the mere idea was unthinkable before.

Binary-XML isn't for everyone. Ant should never use Binary-XML. Nor would it really help XHTML. But when bandwidth is a limiting factor (say... a smart-phone on a metered data-link?) it may just be the thing to save you a few bytes without the huge implementation/test cost of a custom binary protocol.

I've heard the arguement: to just wait a year or two, the bandwidth will come. Rubish. Bandwidth does not obey Moore's law. Take cell-phones as an example. We in the US are just now getting 3G data services! It gets worse. Cell bandwidth is like Cable-modem's. Everyone on the block shares a fixed available connection, but with cell-phones, it is shared per tower. As more people start using data services on their phones, the effective bandwidth will drop because the available bandwidth to the towers is limited. Limited bandwidth is already a problem for some. Nothing I've read indicates that this is likely to change anytime soon.

The W3C's Efficient XML Interchange group (of which my employer is a member) is evaluating a number of proposals (including one from my employer). I have personally seen how our format can produce messages that compete in size with custom-designed formats with a fraction the effort, while preserving XML's flexibility and extensibility. It will take a while, but an efficient Binary-XML will remove the need for many custom formats. Having a standard with implementations integrated into the major platforms will open many new doors.

Gillmore and not using Microsoft

2006-10-17T14:39:00.000-07:00

The latest Gillmore Gang includes Steve Gillmore stating that people choose not to use Microsoft Windows because of lock-in. That may be true amongst the technical elite, but I've watched a whole series of people choose Apple not because they don't like Windows, but because the Mac looks better and 'feels' better. Thanks to the Internet, computers and their operating systems are now a commodity. Macs running OS X look more friendly. Many people just use a computer for a web-browser and music/photo management, and anyone can do that with either a Mac or Windows. (Note: I don't include Linux in that category... yet)

10+ years ago Microsoft worked so hard to build Internet Explorer to compete with Netscape because Microsoft management was worried that the internet would kill the Windows platform. It has finally started to happen, at least for the casual-user home PC. Microsoft should be very scared. Not because people aren't falling for the lock-in, as Steve implied, but because there is no lock-in for a growing class of users.

We Shipped!

2006-10-03T07:57:00.000-07:00

AgileDelta, the little company I happen to work for, just shipped our first commercial release of Efficient XML!. If you are running into problems with XML's verbosity, and compressing your data is either too slow or not performant enough, our Efficient XML product may be just what you need.

Java.Util.Zip bugs

2006-10-02T14:05:00.000-07:00

I've spent much of this morning trying to track down what appears to be a bug in Java.Util.Zip.Deflater/Inflater. The repro is relatively easy:


  byte[] data = new byte[] {
    (byte)0xCF, (byte)0xCF, (byte)0xCF, (byte)0xCF, (byte)0xCF
  };
  ByteArrayOutputStream baos = new ByteArrayOutputStream();
  Deflater deflater = new Deflater(Deflater.BEST_COMPRESSION, true);
  DeflaterOutputStream dos = new DeflaterOutputStream(baos, deflater);
  dos.write(data);
  dos.close();

  ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
  Inflater inflater = new Inflater(true);
  InflaterInputStream iis = new InflaterInputStream(bais, inflater);
  int count = 0;
  while (iis.read() != -1)
    count++;
  System.out.println("count: "+count);

The values for the data array are specific. Other values also demonstrate the error, but most values work without issue. You can also fix it by not selecting the 'nowrap' option, but that dramatically increases the output size for small inputs.

I'm not sure how to report the issue to Sun, so I'm blogging it here.

Java API whine

2006-09-28T15:08:00.000-07:00

My pet peeve of the day: I want a new method added to array objects.

Object resize(int newSize)

Ideally this would be something like

T[] resize(int newSize)

but Java's templates don't really provide support for that that is worth it... so I'd prefer the first. While I'm complaining, I'd like Microsoft to add it to .Net as well.

Why? Because when you care about performance, ArrayList/Vector just don't cut it. I've seen some dramatic performance improvements by moving from ArrayList to object arrays. And when the value is something like 'int' which isn't an Object, you have no choice anyway. At least .Net Generics got that right. If that resize method was there I could replace code that looks like this:

... important stuff ...
... important stuff ...
if (index == array.length)
{
   Foo[] temp = new Foo[index*2];
   System.arraycopy(array, 0, temp, 0, index);
   array = foo;
}
... important stuff ...
... important stuff ...

with this:

... important stuff ...
... important stuff ...
if (index == array.length)
   array = (Foo[]) array.resize(index*2);
... important stuff ...
... important stuff ...

Some people will complain that I dropped the curly brackets, blah blah blah. That resize in only there because the builtin libraries are slow! It isn't topical to the code and hand and should be a small and insignificant as possible. That simple resize() method allow me to write it in a way that is out-of-the-way. That is how it should be.

I can dream, can't I?

The future of the Wiki

2006-09-15T10:48:00.000-07:00

If you haven't seen it, System One is the most impressive attempt at evolving the Wiki I have yet seen. I've had a long history of interest in leveraging computers to augment out abilities. Some old examples I played with were Remembrance Agent and Dashboard. GMail has some similar idea, although it is mostly used for advertising.

Google has shown how powerful a simple search based association can be. System One applies to personal productivity what Google applies to advertising. I'd love to have the same functionality for email, etc. Even better would be something equivalent for software development... One of the tricks that they still need to figure out is how to manage the UI. The current era of these UIs are very mouse heavy. I want to be able to get those benefits without leaving the keyboard. That would really make my day.

Microsoft Zune Announced

2006-09-14T21:04:00.000-07:00

After months of buzz, Zune was officially announced today. Lots of mixed review, as could be expected. The most interesting coverage I found was Engadget's interview with J. Allard. It wasn't until reading that interview that the WiFi and music sharing functionality started to click for me. I'm a bit of a music freak, and love sharing recommendations with people. The 3 Plays or 3 Days bit is brilliant. My question, is when the song is deleted, does it leave a trace behind? I'd love if it remembered what song it was, so I could go and buy it. I need to read more about the Zune store before I consider buying anything though. I'm picky about sound quality and DRM... If it is all play-for-sure... I'm just not interested.

Peter Write says bye bye Microsoft

2006-09-12T17:58:00.000-07:00

So painfully true... I sympathize.

Windows Application Icon

2006-09-10T20:31:00.000-07:00

I was working on cleaning up my app-runner to share out the source code and have run into a problem. I can get the window and exe to have my custom icon, but when I alt-tab, the default Visual Studio app icon is there. How do I replace that with my icon? Do I need a specific size/depth icon? Anyone else encountered this?

Microsoft Exec Rewards

2006-09-01T12:04:00.000-07:00

I remember hearing about this via MiniMicrosoft, but now it is official. From the Seattle Times:

Microsoft's top 900 leaders got almost $317 million in company stock Thursday, the first installment in a compensation plan meant to reward them for the company's performance during the past three years.

As an ex-Microsoft employee and current shareholder, all I can say is WTF?

Lets see... what has Microsoft done recently: Vista is collosally late, over-budget, and under-delivering. WinFS came, randomized all of SqlServer devision, cost the company untold dollars and was then quietly put to pasture. Visual Studio 2005 did finally ship but was also late, and is still just playing catchup to Java community. Avalon, the next generation of application has went through countless re-architectures, re-orgs, and has become little more than a better WinForms. MSN has done well, but the MSN/Windows-Live naming is a total fiasco, and well... HotMail still sucks.

So what did these executives do to deserve this compensation? I'm sure some of them do truely deserve it, but as a whole, this just further proves to me that Microsoft is a sinking ship. (A-la IBM circa 1995...)

Toying with Vista

2006-09-01T11:11:00.000-07:00

I installed Windows Vista on my home PC the other day. (I've always loved playing with the latest and greatest toys, so it is surely a sign of my age that it has taken me this long.) I've only just played with it a bit, but enough to form some definite opinions.

I'm not a huge 'Aero' fan. It feels gratuitous without really providing much useful improvement. I'm not against updating the UI, but the goal should be improved clarity and usability. New eye-candy should surve a purpose. As an Apple PowerBook owner, I definitely agree that XP's UI needs an update when compared against OS X's Aqua UI, but I'm not sure that Aero really hits the sweet spot.
The new Start menu with it's built in search is excellent. I'm not sure it will convince me to abandon my little 'Runner' app, but if it were there in XP I doubt I would have been motivated to sit down and write Runner.
Yet another Control Panel reorg... sigh... I feel like every major Microsoft OS release includes some significant changes to the Control Panel. I haven't used Vista enough to know if it finally brings things up to OS X levels of simplicity and usability, but I doubt it... so why change so much?

As that machine is really just a file store and iTunes server, I don't have much use for improved search, or many of the other changes. I'll write up further opinions when I get time to try out Vista some more.

Summary:

When Vista ships, I'm not likely to be in any rush to upgrade. If I buy a Windows laptop when I get around to my next laptop upgrade cycle, I'll probably be forced to buy Vista, and that certainly won't kill me, but I doubt I'll be clamoring for any of its new features. I still think I prefer OS X's simplicity and consistency (and useful eye-candy, such as Exposé) on one end and Linux's extreme flexibility at the other end. Microsoft is stuck in the middle, by the nature of business/enterprise market pressures.

Microsoft, where's the love?

2006-08-31T22:25:00.000-07:00

Ahh... to reminisce for the days when Microsoft cared about developers. Remember Steve Ballmer's 'Developer' chant? I certainly do.

That is why I was so surprised when I went to renew my MSDN Subscription, and the online tool wanted me to pay almost $11,000!!!! (Due to a change in subscription model, the online renewal tool wanted to charge me full price, rather than the update price, what was only $3,500...) I know MSDN Universal is chock-a-block full of goodies, but that is ridiculous! Hm... so part of the change is to tie MSDN Subscription to a Visual Studio product skew, either Team System or Professional. Then you can pick an MSDN Subscription that includes everything, or OS releases but not Office software.

Now MSDN has always been expensive. It should be, since it includes all sorts of great perks, like licenses to install and try out almost every Microsoft product (for development purposes). Still, I find the current pricing to be outragous. This is just further proof that Microsoft is all about the enterprise. Microsoft has given up the small guy to Linux and Apple. What other explanation could there be? Yea, I know about the free Express skews of Visual Studio. Hello, have any of you used Eclipse or NetBeans recently? Or how about a fully decked-out emacs development environment.

Development tools is not where a company like Microsoft should be trying to make money. (I have one pet theory... I think the Developer Division inside MS has gotten so bloated that they need this money to keep the hideous monster from collapsing horendously.) Microsoft should be making great development tools that make people _want_ to develop on Windows. If you are developing on windows, you are more likely to focus on developing for Windows as your target platform as well. Microsoft should be releasing C# development tools that make the Mono crowd want to develope in VS, even while their target is not Microsoft's CLR.

I'll pick out my two biggest pet peeves, after having spent most of the last year using Eclipse and NetBeans:

Unit tests - Why isn't NUnit just built right into the C# VS experience?
Profiling - Why do I need to spend $2000 to get a profiler?

The problem is Microsoft is building everything for the enterprise developer. I don't blame them, that is where the fat licensing royalties come from. But hasn't anyone over there read Innovator's Dillema? Wake up! All you are doing is playing catch-up to your competitors. They keep nibbling at your user-base. Keep that up and eventually your user base will disappear.

I think it is already starting to happen. It has for me.

Sprint/Palm/Windows ... it's all about the phones

2006-08-28T10:33:00.000-07:00

After this last week of vacation with zero Sprint coverage and 2 years of spotty coverage at my house, I think it is time to switch providers. I've been with Sprint for over 5 years, probably past 6 years, but I'm sick of bad coverage at my house (Calls are often routed directly to voicemail when I'm at home) and a pathetic choice of phones. My girlfriend has T-Mobile and they look like they have the best plans, especially if you want data connectivity.

I've carried around a Treo for 4+ years now and never regretted it. If you need to track appointements, todos, contacts, and have a great phone, the Palm Treos beat anything else I've ever touched. I recently tried out the new Treo 700wx and have to admit that my exit summary was that I would not want to buy the device. I'd pick the Paml OS version every time. I don't care that Windows Mobile has better email connectivity and can view my Excel and Word documents. The usability of the device was so bad compared to my old Treo 600 that I was actually looking forward to returning to my old device. Windows Mobile has a long way to go before I recommend it to anyone who isn't a gadget freak or likely to really benefit from it's improved Windows interop.

Now I just need to figure out what phone to get. The Samsung sliders are near the top of my list (that is what my girlfriend recently upgraded to and it is sweeet!) but since I'm going to have to break my contract with Sprint, I'm looking for something inexpensive (with a contract). It should have decent j2me support, and ideally be useable as a data modem. I like the Nokia 68xx series, but no-one is carrying the current versions of that form-factor (and the older 6800 is showing it's age)...

Back from Vacation

2006-08-28T10:05:00.000-07:00

I just returned from a week long vacation with my family in Maine (across the bay from Bar Harbor). As always a week is never enough, but that was all I had time for. Possibly the best part of the vacation was spending time with my little nephew. At 18 months, he provides a constant source of insight into how we learn. He has just learned to walk, and isn't talking yet. While I was there, I watched his ability to navagate stairs improve noticable. Little tikes like him are amazing to be around... Especially when you have someone else to hand him over to when he is cranky or needs a diper changing. Also, keeping up with him as he scurried about was excersize in and of itself. Spending time with a little guy like him, and living in a comparatively primitive cottage by the shore, makes me realize how much of the hubub of my everyday life really matters in the grander scheme.

My flight over wasn't such fun though. A red-eye into Detroit (<4hr sleep) followed by a cancelled connecting flight left me a lot of time to work through the beginning of my Ruby on Rails book and read Oryx and Crake. Both highly recommended. In general the staff at the detroit airport was very friendly and helpful, and the airport has the weirdest tunnel I've encountered anywhere. Definitely odder than Chicago. WiFi was available, but at ~$10, it was as overpriced as everything else there.

Start Menu Mess

2006-08-17T09:47:00.000-07:00

As one might imagine when reading some of my recent posts (Not quite Quicksilver, and Quicksilver) my start menu on my main machine is out of control. I can't find anything anymore. I've long been a fan of the Quick Launch bar, but that steals precious pixels.

So I decided to be a good geek and code my own solution. It ain't beautiful, but it ain't ugly either. Best part? Only took a few hours , a good part of which was spent fiddling with silly things like the icon editor. It is hard-coded to look at the All-Users start-menu, the current-user start-menu and the current user's Quick Launch bar. It builds a list of all entries and lets your type just part of a name and sorts the list of entries based on what your have typed so far. When you type or double-click on an entry from the list, it runs whatever you selected, then hides itself. It is hooked to Window-, so you can run it from anywhere/anytime.

Here is a screen-shot:

Best part? C# + WinForms + P/Invoke mean that it only took a few hours to build. The VS 2005 WinForms builder is vastly better than what was available before. DataBinding in the ListBox control made binding my sorted list to a list-box trivial. P/Invoke meant that I could easily invoke the necessary Win32 APIs for ShellExec or to bind to Win-Space. Developer productivity baby!

There are still lots of gotchas... It doesn't resize. It is 2006 and the UI layout tools in VS _still_ doesn't generate code for an automatic resize. Delphi had this down in the 90s folks! Why doesn't VS ship with a Win32 P/Invoke library? Shouts out to Http://pinvoke.net, but I often find that the code provided can only be used as a template. MSDN and Microsoft User-Education should be all over this... P/Invoke and a standardized Win32 CLR library would make writing sample code for Win32 so much better. C#/CLR may be here to stay but Win32 ain't going away, either.

It still has some kinks to work out, and the code could use some clean-up. The biggest issue I'm trying to track down is why it beeps at me when I type <escape> or run an app... Not much of a problem, since I often run with sound off, but I'd like understand why it is doing that.

Hopefully I'll get all that figured out next week while on holiday visiting my family, and then I can post the source.

Managing for your career

2006-08-15T12:44:00.000-07:00

I was just reading Greg Linden's comments on Joel Spolsky's latest essays on management ([Command-and-Control] [Econ 101] [Identity]). It reminds me dearly of why I left Microsoft. I saw a number of managers do an excellent job of managing their careers to the detriment of their team. Not all of them realized what they were doing, but some definitely did. I think you will find similar problems at any large company.

One aspect that neither Joel nor Greg really touches on is that Identity management can be hard. In many ways, top down management is what people expect, and to involve people more proactively can take some convincing. A lot of people have been conditioned to accept top-down management as the way things work.

(note: this is my second trial post with Windows Live Writer...)

Not quite Quicksilver

2006-08-14T15:40:00.000-07:00

I've spent a few hours here playing with some of the Quicksilver-like tools that Kristian Kristensen pointed me at. Some are interesting, and I tried Slickrun and Launchy but I'm not impressed. I'm looking for one simple feature: I want it to scan my start-menu (and a few user-contrib directories) and make it easy for me to find entries. The launchers I tried didn't let me control where it scans (at least not that I can figure) which is a crucial for me.

I've heard that Vista improves the start-menu. Will that solve the problem?

Dev vs Test/Ops

2006-08-09T09:09:00.000-07:00

Dare's post 'Replacing Operations with Developers' reminded me of many discussions I had at Microsoft, especially when products were slipping and features were getting cut. A friend of mine once opinioned that developers are optimists and testers are pessimists. Developers (esp at Microsoft) tend to be smart, ambitious, and believe that they can solve any problem with clever coding. This has lead to some horrible situations, when developers believe that other parts of the team are holding them back.

Part of the problem stems from the nature of the task. Developers often a working with development plans that involve dedicating days or weeks to individual problems. Testers and Operations tend to work in terms of hours. If most of your day is spent task-switching and the rest is spent fire-fighting, you have a problem, and I have seen this happen repeatedly in Test and Ops teams, due to over-commitment or bad management. (I've seen the same happen to dev teams, but it is slightly less common.)

Another problem that sometimes arises is simple hubris. Developers like to think they can do/build anything, and so often look down on the Testers and Operations staff. This serves no-one and often exacerbates the problem.

But that isn't to say that Developers should not be actively involved in test and lab operations. Developers must understand what Testers and Operations have to deal with before they can honestly evaluate what these critical staff are really contributing. Test driven development does not mean no test staff. It means that the test staff can do really test integrations, back-compat, etc... all the complicated tests that test that all the pieces fit together like they are supposed to. One of my favorite things I saw happening at MS before I left was a move to Model-Driven-Testing. This required a lot more of the Testers, but resulted in some of the best bug reports. Even better, it got the testers writing non-trivial code and doing something which really felt rewarding. API testing is monotonous. Data-driven-tests are often worse. A good test team is one that is excited about what they are doing and finding bugs that make a Developer scratch their head. This only happens when the code that the Testers are bashing at is already solid.

All of this also depends heavily on what are your requirements. At Microsoft, despite what people may say, the targeted quality bar was very high. Any bug that gets through may mean a painful back-compat hack in the next revision. If you aren't working on a project that has such stringent requirements, then maybe TDD can mean the end of an explicit test-team. Honestly though, anything that takes more than 5 dev-years to implement (yea, I pulled that number out of my arse) probably needs a dedicated test team.

All this applies just as well to Ops. A friend of mine works for network ops for a tech firm that works with banks. He can barely write script, but I wouldn't want to replace him with a developer. He spends his days trouble-shooting network integration issues with customers. How many developers do you know who know anything about IBM mainframe networking? He spends days working with customers to troubleshoot the worst issues. No amount of tools or automation will make his job go away.

I guess one way to think about it is that Developers leverage working in a world of limited variables. To do their job, they need to constrain the inputs. Testers job is to be creative and introduce some new variables, they can expect a customer to experience. Operations live in a world of near infinite variables where half the job is just figuring out what the variables really are. The tricks that make developers so effective at writing code, do not necessarily transfer to these other disciplines.

Zooommmmmm

2006-08-04T14:17:00.000-07:00

How am I supposed to focus on work when I have Blue Angels doing flybys? And such beautiful weather! I just want to sit back in the sun with a book and a drink... Vacation, why are you 2 weeks away?

TGIF.

QuickSilver

2006-08-04T11:03:00.000-07:00

As anyone who reads 43 Folders knows about QuickSilver. I have it installed on my PowerBook and find in amazingly useful, even if all I normally use it for is as a launcher. Since I spend my day in Windows-Land... I'm wondering if anyone has made anything remotely similar for Windows?

I started looking at building something like it myself, and that is still my fallback plan. That would be a good excuse for me to spend some time with C# and WinForms. After some recent work that had me building a Java Swing app, I'm curious to see how WinForms feels in comparison.

PowerBook keyboard woes

2006-08-04T10:45:00.000-07:00

Long ago, I wrote about my frustrations with the Fn key placement on my PowerBook. Someone has apprently been Googling around, found my entry, and then left a note that DoubleCommand has had some significant revisions since then and now allows you to reassign the Fn key! One more reason why I want a MacBook... I keep saying I'll just wait for the Meron based machines to come out and settle in a bit. I due hope that this next batch will resolve some of the heat issues that the first generation of apple-intel laptops.

Less Rambling Venting about Developing for .Net

2006-07-28T10:04:00.000-07:00

Dare's comment on my very rambling post yesterday made me realize how rambling it was. I had hoped to post a follow-up last night, but didn't quite find the time. I had a few points in my rambling vent.

NDoc is dead, and that saddens me... this lead to my venting points 2 & 3
Lack of developer community for .Net - the primary proof of this (for me) is the lack of open-source tools.
Lack of developer community is directly tied to Microsoft's insistence to limit .Net to Windows.

The .Net platform is a great platform. It is fast, and C# is a briliant compromise between the tensions of easy-of-development and developer-control. What I have noticed in spending most of the last year working in Java, is that despite this, I prefer to program in Java. So I ask the question why? NDoc's demise is related to some of my answers, so I slapped it all together in a single entry. Back to my points.

(1) NDoc is Dead: NDoc died due to lack of development support. Too much work and too few developers contributing. Version 2.0 update to the framework included some significant changes. Generics aren't just something slapped on. They are the result of man-years of thinking, planning, and work. They also have nontrivial impact on something like NDoc. I'm not surprised one guy can't just bang out an new version of NDoc that adds support for all the new v2.0 features. Microsoft made things worse by announcing Sandcastle which removes much of the need for NDoc. Take a project that already has had a hard time finding support and announce a free compentitive prodoct from Microsoft and you can smell doom in the air.

(2) Lack of developer community for .Net: One of the things that I love about Java, is that there is some many great open-source tools, and an excellent culture of support has developed around sites that support those tools. Back in the olden days, when I had a IBM XT clone and a 1200 baud modem so I could log into the local BBS, there was a great collection of tools for MS-DOS. Between then and now, the Microsoft development community has become very business oriented. There are some great tools out there, but they are proprietary. Go on SourceForge and look for C# projects. It seems that Microsoft has created a culture that encourages people to create and sell their custom ActiveX grid-control. What is missing is the college and graduate students creating a cool new tool as part of their thesis. There are a few exceptions, but they are rare. Coming back to work on C# developement, the lack of tools (compared to Java) is palpable and irksome. Microsoft makes this worse by failing to leverage the community when extending VisualStudio. Does VisualStudio ship with NUnit support? No. NAnt? Nope. Show me one (non-commercial) external tool/standard that Microsoft didn't develope that has been integrated into VisualStudio. I can't think of any.

(3) Lack of developer community is directly tied to Microsoft's insistence to limit .Net to Windows: So why are people creating cool tools for Java, but not .Net? I can think of 2 reasons: (1) Java has been around much longer. (2) Cross platform support. #1 is obvious and there is nothing that can be done. #2 is where I think Microsoft is shooting itself in the foot. For a long time, Microsoft needed to leverage tie-in as much as possible to realistically create Gates' visions of a PC on every desktop. It was necessary to keep prices down and avoid fragmentation. That vision is accomplished, and now we are on to new things. Time to let tie-in be less of an issue. Multi-platform networks and cross-platform development are the normal now. Accept it and move on. Do you want C# to out-shine Java? Let your child have wings and be free. What is good for .Net is good for you. I'd be much more interested in using C# if I could use it on my Mac PowerBook. Sure there is Mono, but there are still issues with UI and most of the home project apps I play with have UI.

I'm not asking Microsoft to Open-Source .Net. I'm asking for a cross-platform CLR with enough libraries that I can write a real application and have some hope of running it on a Linux box and a Mac box. Given that and a decent, cros-platform C# IDE, and I think the community around .Net would improve dramatically. If Microsoft then showed a hint of interest in working with the community, rather than just producing it's own independent clone of every successful community tools, and the .Net community would explode.

NDoc demise

2006-07-27T10:25:00.000-07:00

According to a number of sources, NDoc is dead. The project owner claims that a large part of the problem is lack of community. I mentioned lack of a supportive community in a previous entry. The overall community is one of the reasons that I actually prefer working in Java vs C#, despite my preference for the C# language.

Microsoft needs to understand that Community is more than just lots of vendors creating commercial components, or MVPs answering questions on newsgroups. The .Net platform is an amazing platform. Microsoft needs to figure out how to make it _fun_ to write tools and libraries. Part of the problem is the lack of Linux/Unix support. Platform tie-in is dead! Give up on your Windows glory days. Windows won the desktop. Linux is not poised to steal it anytime soon.

Microsoft needs to stop weighing down it's products with 'tie-ins' that just hurt customer adoption. XBox is a great example. XBox did as well as it did because it leveraged existing Microsoft assets without crippling itself. The XBox OS is based on Windows NT, but they stripped it down to the bare minimum. Compare that with Windows CE/Windows Mobile? Have you ever done a true head-to-head daily-usage comparison of Windows Mobile vs Palm? I have. I'll take Palm every time, despite Windows Mobile's greater choice of applications and richer platform. Why? Because it's my phone. It should be just as easy to use as a phone as my girl-friend's Samsung phone. Windows Mobile just feels clunky, less intuitive, and less polished.

I forget who was posting those love reviews of Windows Vista with all the complaints about fonts and different UI control styles, button placements, etc... but his comments capture why Mac laptop sales are up, why the IPod is such a run-away hit, and also why I prefer to program in Java. Its all about the little details.

Microsoft needs to step away from it's long habit of tying everything back to Windows and Office, and making that a primary selling point, and instead make great products that integrate well with Windows and Office. The difference is subtle but key. I can't wait to see how they package Zune. Will it be chained at the ankles with Windows leashes, or will it be proof that some people in Redmond can see beyond that?

Anton Lapounov is blogging!

2006-07-26T13:43:00.000-07:00

One of Microsoft's XSLT best experts is spreading the word about XSLT in System.Xml. I worked with Anton on and off for a number of years and have immense respect for him. It is great to seem him educating people about System.Xml v2's XsltCompiledTransform, which he had a huge part in delivering. This was a huge effort by a number of people and it should be of great value to anyone who need to execute XSLT on a windows machine.

Power Tool Races

2006-07-24T13:53:00.000-07:00

I spent Saturday getting sunburned and helping out Hazard Factory host Power Tool Races in Georgetown (a neighborhood in south Seattle). Insane fun, insane heat, and power tools racing at breakneck speeds. What could be better?

Java vs C# .. take 42

2006-07-08T17:52:00.000-07:00

I just spent the last few weeks back in the depths that are C# development with VisualStudio. I've spent most of the last year doing Java development in Eclipse (after almost 3 years of C# development at Microsoft) and bouncing back and forth has really called out some of the differences between the platforms and communities.

Lets start with tools: VisualStudio... VisualStudio is a mammoth application, that predates the Java platform. Modern VisualStudio installs are huge and include an immense set of tools and documentation. The downside is that having so much legacy and such a huge feature set, the application no longer feels cohesive. Despite having many of the same features and being more responsive than Eclipse (since it is mostly c++ code), VisualStudio comes across as less modern, less current. Also, strangely, VisualStudio development seems to sap battery much faster than Eclipse development does. The plug-in community is virtually non-existent. I tried to find CVS (and Subversion) support, and found nothing that didn't look like a weekend hack. There is no real community around extending VS, although there are a number of commercial vendors whose products integrate. As VS is a C++/COM app, building extensions is more complicated than extending Eclipse.

Eclipse also suffers from cohesion issues, but for different reasons. The 'perspectives' modalities feels like it creates more problems than it solves. There are annoying pauses as the GC tries to keep up with the UI. The live compilation is amazing, and the run/debug menu is possibly my favorite recent IDE innovation. I find the development environment noisy and the context menus over-busy, but basic functionality is easy to adjust to and amazingly productive. Also, the plug-in community is lively and moderately organized, and as Eclipse is a Java app, extension development is much easier and more productive than VisualStudio. Finally, the CVS support is excellent and the Subversion support is comparable.

One of the big difference I notice between the 2 communities, which is also reflected in the tools themselves, is how open they are. Eclipse is open-source, and actively encourages others to develop extensions. VisualStudio is proprietary and the SDK for building extensions is not well documented. Eclipse is all about anyone and everyone being able to build an extension. VisualStudio is all about companies selling their extensions.

Java's community has included a strong open source trend since before open-source was even trendy. The java community has produced some very strong community sites that support the Java developer. Microsoft's C# platform has never managed to engender the communal spirit that Java has. Part of that can be blamed on C#'s relative youth. Partly it is the result of a general Microsoft attitude to prioritize the corporate client. Eclipse does nothing to stop someone from charging for their extension, but it makes it so easy to build an extension that the bar is set rather high for a commercial entrant. The strong community has also meant that developing an good extension can be an excellent career strategy. Microsoft may have it's MVPs, but I feel that the community just isn't there. Microsoft products aren't designed to make you want to extend them, they are designed to themselves be the end-all-be-all.

The net effect, is that I prefer java development. This surprises be, as I prefer C# as a language, and I find many of Microsoft's libraries easier to use than the Java equivalent. Eclipse just feels friendlier and more open to me. If I didn't already have too many unfinished projects, I'd be contributing to one of the C# extensions. But that is part of the draw, the open-source lure. "Don't like it? Come help make it better!" There are a number of things I'd love to tweak in VS (mostly copying Eclipse) but ... well, I can't. Worse... I really just don't want to. I did some C++ development recently and after spending an evening tracking down a crash related to a typo while implementing a custom resizing array, I came to remember how productive I am in Java and C#. Sure I can write code that has 2x faster inner loops in C++, but it takes me 4x as long to get it all working!

As a close, here is my feature request to the VS team:
- 'run' menu like Eclipse. I often am building cmd-line apps to test an algorithm or library and need to be able to run multiple configurations. I love that this is just 2 clicks. In VS I need to get to the debug properties page for the project and change the args each time.
- Better interactive error reporting (background compile). The C# gives me red squigglies for some things, but it seems that there are still _many_ errors it does not catch.
- Faster build. This ties to good incremental errors. The build should be happening incrementally in the background all the time with only the final link steps happening when I invoke 'build' or 'run'.
- Better help integration. This especially bugs me when I'm developing win32 code. There are often multiple hits for a given keywork, but MSDN picks one and just jumps there. It should give me a choice, and it should remember what I've lookup up recently, so that when I am doing Win32 development, it defaults to the win32 match, and when I start doing C# development it should just start noticing and default to the CLR match. Currently, I can manually do this by selecting the appropriate filter in MSDN, but since I often program in C# during the day and win32 at night (on the same machine), that doesn't really solve my problem.

one apartment and six drummers

2006-07-05T10:57:00.000-07:00

I've always been a sucker for rhythm: Music for one apartment and six drummers.

via jwz

To Switch or not to Switch, that is the Question

2006-06-20T10:37:00.000-07:00

Last week I read Tim Bray's post about potentially switching to Ubuntu. That night I stopped by a computer store on the way home (actually as a way to avoid sitting in traffic...) and swapped out the hard-drive on my Dell D610. I downloaded the latest Ubuntu install ISO and gave it a whirl. I must say that the Linux installers have come a _long_ way from what I used back in '96 or even just a year to two ago. That really got my expectations up. The my first install hung. It just sat they trying to install gnome_panel for a hour or two. I tried again. It worked!

Mostly... I still can't get the wireless to work. The driver finds the device just fine, but it defaults to not-activated, and no amount of fn-key twiddling, rebooting, or bios prodding could get it to activate. I was impressed that I could just slap an pcmcia card in and in minutes had wireless up and running. That get me smiling again.

Then I found that suspend didn't quite work (my trackpad didn't work on resume).

I've swapped back my Windows drive, since it has my email, etc on it.

It looks like I may need to change the minipci wireless card (or live with a pcmcia card) to get wireless running reliably under Linux thanks to Broadcom's reluctance to document their chipsets. How compatible are the various mini-pci cards? Is it really a standard form-factor (i.e. can I buy any mini-pci wireless card and replace the Broadcom card?) From one oneline site I found, it shows an Intel mini-pci card that appears to be longer and skinnier than the pictures in the Dell service manual.

Maybe I should just get coLinux working fully and live with that until my next laptop?

MSBorg and Bill...

2006-06-20T09:43:00.000-07:00

Unless you have been living under a rock, you know that Microsoft announced that Bill Gates is stepping down. Being an ex-Microsoftie, and a techie living in Seattle, one can't help but have been talking about it.

Microsoft has lost its luster. It is bogged down in a middle-management morass of mediocreness. Microsoft's main business is focused around Windows and Office. Both have been so successful as to be their own worst enemy. How can you sustain growth when you are the dominant player in a limited growth market? XBox is promising, but that market is so competitive, I don't ever see it being the cash cow that Windows or Office is.

My dream (as a Microsoft stock holder) is that Microsoft accept that Windows and Office are commoditized and move on. Stop trying to tie everything to Windows. Leveraging Windows is one thing, but be sure to avoid shackling every new venture with a Windows legacy. One of the failings of the Windows CE platform is that it was designed as Windows-lite. I don't want Windows on my phone. I want my phone to be a phone and have good integration with my Windows apps. There is a huge difference between the two. Just read the reviews of the Palm Treo 700w vs 700p. Every comparison I have read favors the 700p. Why? Windows CE has more functionality, and some amazing apps. Palm has a simple platform that is targeted for what one actually uses the device for.

Windows Live (aka MSN? Microsoft branding has never made any sense to me...) has some interesting potential. One of the reason I believe Google has done so well is to leverage an amazing service and storage infrastructure to build new applications. In order to deliver MSN/HotMail/MSN-Search/etc, Microsoft must have some level of similar infrastructure. They need to unify and leverage that in the same way that they leveraged Windows and Office to propel growth through the 90s.

Presenting your service on the web has a huge advantage over traditional desktop apps: accountability. If some group releases a new Google Beta, management can trivially watch add revenue directly related to that new venture. Web logs can produce incredibly useful data about how many new users that beta is bringing into the Google fold, how much revenue, how many click-thrus on the adds. When a similar group add a feature to Windows Vista, how does management know if anyone even uses it?

Microsoft will never again be the company it was. Time does not flow in reverse. But Microsoft is a company to be reconned with, and I can only hope that the recent management shake-ups will lead to a slightly more nimble homunculus.

XmlLite

2006-06-15T14:52:00.000-07:00

I just saw Oleg's annoucement about XmlLite! XmlTeam Blog... you should be all over this! I contributed a bit to this (in many ways), and can definitely say that any 'native' C/C++ MS developer should be taking a peek. Basically, it is an System.Xml.XmlReader-like API in native code, with a blazing fast parser underneath. It also provides lots of hooks for controlling entity resolution, memory allocation... everything that a system's level developer would want.

Acting like the Evil Empire (we know and love)

2006-06-14T21:30:00.000-07:00

Microsoft recently released Beta 2 of Vista. but the MS download servers are overloaded. So two enterprising souls published a Torrent of their download. Why Microsoft doesn't embrace BitTorrent for their downloads is beyond me. As an MSDN Subscriber, I would love such a feature. So there are some people out there who are ahead of the Microsoft curve (not that is is hard to outrun a glacier). How does Microsoft reward them? Cease and Decist. It just makes you feel that might really still be the Evil Empire of yore.

(via Scripting News)

Java: Integer.toHexString()

2006-06-14T11:40:00.000-07:00

So who out there would expect this code to work?


    void write(PrintStream out, int x) throws IOException
    {
          out.println(Integer.toHexString(x));
    }
    int read(BufferedStream in) throws IOException
    {
          return Integer.parseInt(in.readLine(), 16);
    }

I sure did. Nothing in the docs for toHexString() or parseInt(s, radix) indicates that it parseInt can't handle the output of toHexString(). How can this not be a bug? But sure enough, parseInt() can't read back in the result of toHexString() if the original value was negative (i.e. the high bit was set).

Why can't I just have unsigned int (and unsigned byte)? sigh...

XQuery ... almost

2006-06-13T13:27:00.000-07:00

W3C just announced XSLT 2.0, XML Query and XPath 2.0 Candidate Recommendations. I remember, back in 2000, betting on when XQuery would ship. None of us expected that it still would not be a finalized reality in 2006! XML was supposed to be simple! Any XML spec that takes that long to finish should be walked away from. Cut your losses and run!

Don't get me wrong, XQuery is chock-a-block full of amazing ideas. So toss out some of the decisions that made it take 7 years to produce a query language which doesn't even include update! XPath had its problems, but the spec was just a few pages. XPath was too simple, and all the database vendors wanted a query language that fit their existing models... so they designed a language that requires man-decades of development to implement!

Given that we are still figuring out what a semi-structured database really is, and how XML fits into the big picture, why is the W3C asking us to commit to such a juggernaught? And can someone please fix the type-system? The XML-Schema data-types is bad enough for text-validation, but as the type-system behind a programming language it is a colossal mistake.

KEXP - The Church of Good Music

2006-06-13T09:33:00.000-07:00

I often listen to KEXP on my drive into work in the morning. They are doing a pledge drive right now, and they closed one plea for donations with the choice quote: "KEXP - The Church of Good Music."

Then John when on to make me happy (and feeling old) by playing a medley of TMBG, King Missile, and Morrissey.

It isn't about AutoBoxing

2006-06-12T14:22:00.000-07:00

I was just reading the latest Dr Dobbs, and there is an article about Java 5/6 and performance. One of the items mentioned was AutoBoxing. This reminded me how this is yet another case where Java copied the wrong thing from C#. C# has AutoBoxing and ain't it cool! Lets be like cool C# and have AutoBoxing too! Arrrgg. I think the problem is that a lot of people collude 2 aspects of C#/CLR. On one hand, there is AutoBoxing. The side effect of that is that 'int' is equivalent to 'System.Int32'. The second part is what is so powerful. I really find it frustrating that I can't do things like 2.toString(). Why do I have to type Integer.MAX_VALUE, rather than int.MAX_VALUE?

To me, this is just another example of how Java is still playing catch-up to C#. I love Java's cross-platform-ness, and I generally find that Eclipse is an amazingly productive IDE, but if I still find myself missing features from C# on a daily basis. I've mentioned some of these before, but the short list of features I miss are:

IFDEF - C# gets this correct. C++ macros are dangerous, but basic IFDEFs are ultimately a requirement of every project I've ever used.
Struct - I want arrays of structures, not arrays of references to objects
Enumerations - Fixed in Java 1.5 (mostly) but most corporate environments will still be targeting 1.4 for years to come.

Since writing my original rant, NetBeans' profiler has been released and is justification to have NetBeans installed on any Java developer's machine. I vastly prefer Eclipse over NetBeans, but for profiling, NetBeans' profiler is competitive with a lot of the commercial offerings costing hundreds of dollars.

I've never tracked the JSR process. Are there any JSR's for IFDEF or Struct? If not, I tihnk it is time for me to try and stir something up.

If anyone wants to get me a present...

2006-06-05T09:44:00.000-07:00

In case anyone with a few million burning a hole in their pocket were wondering what I might like for a present: http://www.wists.com/chateaux.html. I'm really not too picky. Most any one will do...

Doctor Who?

2006-05-15T10:31:00.000-07:00

I finally got around to renting the new Doctor Who series. I'll admit to having been a huge fan back in my teens. I used to make my poor mum hold off dinner until after the show because I couldn't bear to miss an episode. I was a bit surprised to find that my better-half had also been a Dr Who fan. Back on topic... the new series is surprisingly good. I've only watched the first 2 episodes so far, but I can't waite to watch more.

It seems that BBC has some sort of copy-protection on it, because none of the programs I normally use to watch European movies work with this DVD. I had to use up one of the prescious region-code changes allowed by my powerbook's DVDPlayer app. I tried using various software that I have used to watch region-2 DVDs before, but none could read the DVD. I tried a package which claims to be able to de-macrovision DVDs, but that also failed. It is sad because this forces me to BitTorrent the shows... I don't have cable because I just don't watch enough tv to justify the cost, and with this I can't rent it on DVD (since it isn't out in Region-1 NTSC, at least not as far as I can tell), so BBC is forcing me to BitTorrent it... when I'm perfectly willing to give my money. This is an example of why copy protection is so problematic.

Thanks MSDN Team!

2006-05-15T10:08:00.000-07:00

One of the best things about Microsoft is that despite it's middling management, it really does have a lot of great people there. After MiniMicrosoft linked to my Visual Studio rant, I was contacted by some people who helped me track down the problem, and asked for some constructive advice on how to improve the documentation. This wasn't just a PR stunt... this was people who really want to improve the product.

I've known a few documentation writers at Microsoft, and bumped into one this weekend. I've been honestly impressed by almost all of them... I honestly don't even know why they stay at their jobs (and some of them haven't). The 'user-education' teams at MS are horribly understaffed, and typically spending a huge amount of time dealing with ridiculous busy work (updating old documentation to new file formats, dealing with buggy, unfinished new tools, etc). Like much of the rest of Microsoft, I think the documentation teams are suffering from middle management that is more interested in building empires and getting a raise than bulding good documentation.

So what was the problem that caused MSDN to crash for me? Embarasingly, I had acquires some malware. I honestly have no idea how I managed to pick up this particular evilness. I'm generally careful about what I install, and believe it or not.. I have never had a virus infection on any of my machines. I know when this hit me, because I noticed something using CPU on my machine and had killed that process and run Windows Defender which had supposedly found and removed the infection. What I found most interesting was that Microsoft actually has 2 malware clean-up tools! Apparently there is also something called Windows Live Safety Center. I don't get what the difference is... It appears that Windows Live Safety Center actually removed whatever it was that had infected my machine. phew. Much as part of me wants to clean my machine, it is embarasing that I feel the need to reinstall the OS on a machine I've only had for 3-4 months. I'll just keep praying for some benevolent deity to drop a lovely 2nd-generation intel Mac laptop... one that won't roast my lap. I still think that a Mac laptop with good PC virtualization would be my ideal platform.

Rane rocks...

2006-05-11T10:52:00.000-07:00

A friend of mine who uses Serato Scratch Live pointed me to an open house Rane was having for their new mixer which integrates the Scratch Live hardware into a basic 2 channel mixer. I popped up there for the event yesterday. I've always liked Rane gear, but I had no idea that they were located so close! I've got an ancient Rane MP22 mixer that I love. The Rane folk were outside welcoming people as they arrive, and any company reps that great you with, "you know, the last to arrive pays for the beer..." is good in my book. The event was simple, direct, fun, and unpretentious. And if you are a DJ, look at Scratch Live. This is the future of DJing.