It is very true that the Indian IT companies are failing to corner big deals in the domestic market. Both the IT companies and the domestic clients are to be blamed for that.BTW, don't take that last line too seriously, I am pretty sure he just forgot to add a smiley ;-)
Let us first start with the Indian clients. Their IT awareness is quite poor. Their business processes are also not very much IT-dependent. In fact most of them view computerization as an additional burden - a sort of necessary nuisance. The business processes of those companies are so non-standard that it becomes rather difficult for them to get the real benefits of IT. Today's well evolved softwares require clearly defined and standard business processes and, most important, discipline as pre-requisites. For example, to get the optimum performance out of a Merc you need proper roads and disciplined traffic. Otherwise its high costs of acquisition and maintenance seem to be unnecessary burdens on the owner.
With the exception of banks, telecoms and a few Indian pharmaceutical companies, almost all other Indian companies across the sectors have those inherent problems. So when they do not get tangible returns from their IT infrastructure they tend to pay substantially less for the same. That is the root cause of all the problems. The Indian IT companies focus mainly on the application development side with a distinct bias for manpower intensive projects. For them the earning per employee - or 'billing rate' in the industry jargon - takes precedence over everything else. Indian clients do not pay more than one-third of what an international client would pay. Add to that the inevitable delays associated with the domestic projects due to various reasons like unavailability of proper infrastructure, stiff resistance from the middle and lower management levels who see IT as a source of extra work load, erratic payment schedules etc.
Instead of staying away from such messy, low-profit (often loss making), difficult to manage projects the Indian IT companies - with the exception of Infy - have been systematically exploiting the Indian clients for training their freshers. The domestic projects are known as visa factories. Because of the strict regulations against import of cheap labour in most of the developed countries, visa / work permit applicants need to have a couple of years of experience as a pre-requisite. How do the fresh engineering graduates get that experience? Mostly through the domestic projects!
So how the MNCs are making money out of the Indian clients? Hardware supply is the key. They make enough money from hardware components - like storage devices, PCs, servers, routers etc. - to be able to cross-subsidise the application software. Indian clients - thanks to their great IT awareness - are very reluctant to pay for something intangible which they cannot touch and weigh! Software is such a component. Hardware, fortunately, does not suffer from that bias.
Before I end this post I want to share a funny but real story of an Indian client with the readers. Around ten years back one of my friends used to work as a free-lance developer. One of his many clients was a typical Lala-type trader. After making big money in trading, he wanted to add some class to his business. So he wanted my friend to develop a software for his company. After toiling for a couple of months and bearing with the paan-chewing, gaali-spewing Lala he finally delivered the software in the form of two floppy disks. The installation and the subsequent setup went off smoothly. The bill of twenty thousand gave the trader a big shock. "Bees rupiya ka floppy de ke aap bees hazar rupiya maang rahe hai? Sharm nahin aati?", he said. It proved to be rather difficult to convince him. My friend tried to educate him first. Then only could he hope to convince him! Finally he gave up. He settled for five thousand. Last couple of years the same friend of mine is in US. He is still a free-lancer. He never goes near any client whose skin colour is even vaguely brown.
Tuesday, March 27, 2007
Why Developing Software for Indian Companies is Not Easy
I was reading an article Indian IT biggies losing out to MNCs, when I came across a very interesting and insightful comment by Chanakya. Yet another instance where a comment on an article is far more interesting the article itself. A couple of years back, I quit my job since I couldn't not handle the irrational pressures involved in developing for certain Indian customers - I quit even without having another job in hand. I know of a very idealistically motivated software company that had created a great ERP product for the SMB market - the last I heard, they have stopped trying to sell to Indian customers and were focusing their sales efforts mainly on the US and Europe. It's not for want of idealism, or lack of technical competence, or domain specific competence that you find very little serious software development work targeted at the Indian market. Most of the the fine folks who have tried are suffering from Indian IT market burn-out, for the reasons for which are so eloquently explained by, Chanakya:
Open Source Development - The Perfect is the Enemy of the Good
Subversion developers Ben Collins Sussman and Brian Fitzpatrick in a talk titled How Open Source Projects Can Survive Poisonous People (And You Can Too) (Google Video) stress the need to avoid analysis paralysis . One of the ways paralysis is caused, explain the developers, is the attitude of perfectionists that makes them indulge in endless technical discussions on mailing lists - this includes seemingly interminable discussions on even minor details; The 'Painting the Bikeshed' analogy by Poul-Henning Kemp illustrates this point. As Ben and Fitz put it in the presentation, "The Perfect is the Enemy of the Good".
I wonder if that's the reason they have put up with WebDAV for so long ;-)
I wonder if that's the reason they have put up with WebDAV for so long ;-)
Sunday, February 18, 2007
Dreaming in Code - Andy Hertzfeld Quotes
There are some great quotes from Mac hacker, Andy Hertzfeld in Scott Rosenberg's book Dreaming in Code.- "My style is to get something going really quick and then turn it into the great thing that is the reason you're doing it. You're not working to have it be run-of-the-mill. You're working on it to do something great. But you need to get it started! The key is getting exciting work going; the rest of it will take care of itself. You're sparking off each other - a virtuous cycle - once you're doing the thing you're there to do."
- "There's no such thing as a typical software project. Every project is different."
- "And in this meeting, we kill the snake - we don't just make plans to kill the snake."
- "I'm the kind of developer who likes to throw lightning rods around. To make a great program there's got to be at least one person at the center who is breathing life into it. In a ferocious way."
- "My way of writing code is, you sculpt it, you get something as good as you can, and everything is subject to change, always, as you learn. But you climb this ladder of learning about your problem. Every problem's unique, so you have to learn about each problem, and you do something and get a better vantage point. And from that vantage point you can decide to throw it out."
- "Code is cheap. But often it tells you what to do next."
- "We all need more glory as designers - to show we can design another great thing. Everybody who has a first success, especially when it's young, wonders: Was it luck, or was it skill ? Well, it's a little of both. If you can do another really great one, it shows the world something."
Monday, February 12, 2007
Mary Lou Jepsen - Inspiring
At age 29, Jepsen found herself suffering from blistering headaches, confined to a wheelchair, and sleeping 20 hours a day. She was just about to drop out of school when an MRI revealed a tumor on her pituitary, a small gland at the base of the brain central to hormone production. She underwent surgery to have the tumor removed and emerged from the ordeal ready to move on with her life. “There’s a stigma when you undergo brain surgery: are you still smart or not? So afterwards I tried to challenge myself to find out.” She finished her Ph.D. in the next six months and then cofounded MicroDisplay, a Fremont, Calif.–based company that manufactures liquid-crystal-on-silicon chips for high-definition TV displays. She left MicroDisplay in 2003, citing “creative differences” with its chief executive, but within days Intel was recruiting her.Her health problems weren’t quite over, though. As a result of the operation, Jepsen’s body now makes no hormones, requiring a rigid schedule of twice-daily hormone supplements to keep her alive. Now that she’s a globe-trotting computer executive for the OLPC venture, the regimen can be tough to follow; last March she went into adrenal shock on board a plane, forcing it to make an emergency landing. (On the bright side, Jepsen reports that as a result of her hormone deficiency, she is unaffected by jet lag.)
That's Mary Lou Jepson, CTO of the 'One Laptop Per Child' (OLPC) project.
Writing vs. 'A Real Programming Job'
Financially, writing science fiction is to writing technical books as writing technical books is to having a "real" programming job.- Leonard Richardson, co-author of Ruby Cookbook and Beginning Python,
in an interview.
How Come ZeroConf is Not Yet Wildly Popular
I have always wondered why Zeroconf adoption has been so poor. Stuart Cheshire of Apple who pioneered this technology, briefly explains one of the reasons in a presentation introducing zeroconf(6:16-7:30) at the 2006 Emerging Telephony Conference.
I imagine a number of people are thinking, well, if it's so great, how come I haven't heard of it ? And, that's an unfortunate history of marketing mistakes - I called it Zeroconf, but Steve Jobs thought that was a dull name. So he called it Rendezvous which is a really good name. But, some other company thought so too and they had registered the trademark.
Then we had a couple of Years of the marketing people and the lawyers doing their stuff. And meanwhile we had a technology without a name. So, in that two years every single printer vendor adopted it. But they didn't have a name to call it, which is why you won't see it on the box. But if you have a printer on your network, an ethernet printer, or a 802.11 wirless printer that you bought in the last couple of years, then it has bonjour in it. And, on the Mac you go File->Print and it's right there in the print dialog; No setup assistant, no wizard. On Windows we can't make it quite that easy. But you can download from www.apple.com/bonjour and just run the setup assistant and it will find all those printers on the network, and configure them for windows for you.
Wednesday, November 29, 2006
FOSS.in 2006 Notes
Notes from some of the talks/discussions, that I attended at FOSS.in, held in Bangalore between 24-26'Nov 2006.
Suparna Bhattacharya, the Linux kernel contributor who works for IBM talked about how kernel hackers are obsessed with making the code simple and the minimalist spirit that pervaded the entire kernel development. She also drew parallels with Buckminster Fuller's idea of ephemeralization - doing progressively more with less. I have noticed that this obsession with internal simplicity and cleanliness is something that is noticeable much more with the better open source projects than in corporate development environments. While it's disconcerting in the beginning, after a while you wonder how development could have been done in any other way. Interestingly Suparna also mentioned that she had used a BBC Micro at school - this tallies with my theory that a lot of good programmers are folks who have had early exposure to computers.
Aaron Seigo, the KDE guy, talked about the planned features for KDE 4 . The Tapioca project aims to make VOIP integration with the linux desktop a reality; SVG support is maturing; The threadweaver project hopes to make writing multi-threaded apps easier; Project Strigi will make desktop search seamless and has a DBUS interface; does meta-data search as well. Aaron also mentioned working with HCI teams to make KDE more usable to regular users - I personally like this idea. The KDE project's UI ideas have always seemed juvenile to me.
Luke Kanies talked about Puppet - a server automation tool that comes with it's own declarative language. Puppet is written in Ruby and is the result of Luke's frustration with existing tools. 'Developers do not think about manageability', feels Luke. He also added that there are many opportunities to add small pieces of functionality. Puppet has a central server, a portable resource layer and agents on the network that do the actual automation work. The custom configuration language can be versioned and changes tracked easily, since it's plain text. The author of Puppet is very interested in building an open community around this GPL'd software and said that it might take many generations to make the tool great, and he would help create those generations. Resources in Puppet could be anything - Packages, Users, Groups - just about any low level system object. Resources can have multiple low-level providers. The platform specific details are abstracted away. Clients or Agents provide the server with facts about the hosts every thirty minutes or so. Puppet uses SSL everywhere. Though it's been around for only a little while, it is already in production in a couple of places in Europe and US. Stanford Univ. is a paying client. There is no windows version yet and I could not help feeling that the lack of a GUI might be a disincentive to most admins.
This is one area in which I have lots of personal interest, and my current job also involves similar work. I chatted with Luke a bit. I think it's high time we had a good cross platform (*nix, Linux, Mac, Windows)open source tool in this area. It's time to kick the pricey vendors out - they waste our time and money, and have no clue about real requirements. Good luck to Puppet and Luke Kanies. Google videos of his talks elsewhere.
PHP man Rasmus Lerdorf gave a great talk on how to build a good community site and optimizing sites to handle high traffic; even though he used PHP as the example, I thought it was quite general. Rasmus appeared to be a bit upset with the loads of criticism levelled at PHP these days and pointed out the people making negative blog posts about PHP do not realize that Wordpress was a PHP application. I am not sure if this is a good defense of PHP, the language has real problems. But I think it's helpful to remember that for a long time and even now, it's a great way to get things done quickly. As he himself put it, PHP is for weekend warriors. Rasmus actually began his talk by hypothesizing about why geeks are so motivated to work on open source software or even do non-programming community work. He mentioned the case of some users on Yahoo Answers who answer 500-1000 per day questions some times and attributes it to oxytocin, the hormone that gets released during human interaction. Even geeks who rarely see daylight and live in their mom's basement, have the need to connect with other humans, and hence the motivation to take part in net communities. Rasmus feels that it's not what people think about you, but rather what they think about themselves that matter.
Rasmus observed that people should be able to improve a community site by catering to their self interest;he mentioned that no worthwhile community can be built without giving up ownership - if you don't give up control, you will never build a decent community of users. I personally think this works for some categories of software/communities and not for all. Dan Bernstein, the author of Qmail maintains total control over the direction of the project, Napster was Shawn Fanning mainly, and I could go on and these were projects that were so useful that the community ignored the idiosyncrasies of the owner. PHP is now mainly community driven, and became so around 1997; Rasmus' code in PHP is less than ten percent now.
He took apart a small PHP app with the help of Valgrind and showed how profiling an application can be counter intuitive. It was extremely interesting to hear this portion of the talk and I was unaware that valgrind could be put to such use; http_load was used to load test the site. He also mentioned the XML support in PHP5 including Simple XML, Yahoo'd geocoding api, and cautioned against using participation gimmicks like paid blogging. Solve one problem, a clean and intuitive UI were other tips from Ramus. Surprisingly he said that he finds programming to be boring and tedious, and his main interest is in solving problems. Slides here.
Christophe Hellwig, a linux kernel hacker tried to motivate contributions to the linux kernel. Among other things I learnt that one doesn't have to sign over copyrights when submitting code to the Linxu kernel. In contrast contributions to FSF projects need to sign over copyrights to the FSF and do some paper work as well. Christophe said that for result-oriented people there are lots of interesting problems to solve. Lots of modules need help. He explained how corporate contributors to the Kernel were often at conflict with the kernel developers due to pressures from the boss. There was a Sardarji in the audience who asked how many years it generally took to become a kernel developer - the audience laughed for no apparent reason ;-)
Andrew Cowie made an energetic presentation about the structural problems facing open source projects. He outlined at length the problems with centralized version control, build systems, bug tracking systems, and how they all combined to make collaboration between projects difficult. He made the very interesting observation about the committer/non-committer divide created by centralized version control systems. I wonder if DVCSs really solve the problem - you could have your own repository, but what if your changes are never merged into the main repository - C'mon stop pretending that there will be no 'main' repository with distributed version control systems. Besides the problem is how do you discover these repositories - DVCSs do not address the question of discovery at all.
The penultimate day of the conference saw a panel discussion by Sudhar 'Thaths' Chandrashekar; the panelists included Atul Chitnis, Kishore Bhargava, Karunkar Guntapalli, Arun Sharma, Frederick Noronha, KDE developer Sirtaj Singh Kang,and another bloke whose name I didn't get. Atul Chitnis who incidentally does not have too much by way of actual contribution to FOSS projects was predictably the most vocal about young people these days, yada, yada and the death of curiosity. He even made some vague connection between the ability to take apart devices and the ability to hack. In fact there is no such connection. Check out Karl Fogel's excellent article . Only in India do such vague assertions fly without fear of contradiction - it is no better than corporate software environments where the 'get things done' types and 'Excel gymnasts' are regarded as being more useful than good programmers. One recent college graduate finally got fed up with 'oh-young-people-these-days' rants and explained some interesting work they had done at college, that helped reduced the cost of the implementation. He explained that they customised the software extensively and the proprietary choice was very expensive. He also felt that Rs.500/- was a little high for college students. Instead of patiently explaining to a newbie, Atul lit on to this poor chap. One of the panelists was very honest and admitted that he had failed in his MCA exams, unlike lots of pretentious desi FOSS folks who are still enamoured of pedigree. The questions were all predictable, but the answers were better even though the pessimistic tone was a tad too much for me. I can't figure out how Atul arrived at the conclusion that things were going downhill - In fact there have never been so many Indian contributors as now. Atul and co seem to think that being well informed and following every single mailing list out there, and latching on to every passing tech fad is what contribution is all about. Well Atul, with my limited experience in open source projects I can tell you that working on nontrivial patches is real work - the kind of work that requires you to put down your head and do real hard work. Next time you feel like making a claim like this, please do some honest research and go through the commit mailing lists and actual dev lists. I know that you are playing to the gallery and pretending to be the sole saviour of open source in India. Just as life is what happens when you are planning, real open source contribution is what happens when non-programmers who are over the hill are talking about the glorious days of tinkering with modems. Honestly I don't know what the OSS contributions of the panelists other than Sirtaj, and Karunakar are.
It was funny that while one the very visible participants who is a Google employee did not miss an opportunity to let everyone know that he was working with the world's largest organized cult, I heard from another participant at the conference that a very respected open source contributor and O'Reilly author had resigned from Google after spending just three months there. A few days back news of another very respected oss contributor quitting Google trickled in. I know this guy and he is not the one to go around advertising his coolie status even if he works for google. Integrity and self respect count for more than grovelling before the benevolent master.
I attended this event after six years. The first time I met RMS. Nat Friedman's behaviour during his presentation was extraordinarily juvenile - he used the F-word so many times during the presentation that I got irritated. I didn't get a good impression and the many twists and turns of the GNOME project since then have only confirmed the fact that it is run by a bunch of juvenile delinquents. The recent Novell-Microsoft fiasco only serves to confirm my initial impressions.
Best comment/observation of FOSS.in 2006
When a member of the audience, strangely enough a Novell employee, asked if it was not possible to enjoy the freedom without getting into license wars, someone from the back (either Luke Kanies or Andrew Cowie) shouted:
Suparna Bhattacharya, the Linux kernel contributor who works for IBM talked about how kernel hackers are obsessed with making the code simple and the minimalist spirit that pervaded the entire kernel development. She also drew parallels with Buckminster Fuller's idea of ephemeralization - doing progressively more with less. I have noticed that this obsession with internal simplicity and cleanliness is something that is noticeable much more with the better open source projects than in corporate development environments. While it's disconcerting in the beginning, after a while you wonder how development could have been done in any other way. Interestingly Suparna also mentioned that she had used a BBC Micro at school - this tallies with my theory that a lot of good programmers are folks who have had early exposure to computers.
Aaron Seigo, the KDE guy, talked about the planned features for KDE 4 . The Tapioca project aims to make VOIP integration with the linux desktop a reality; SVG support is maturing; The threadweaver project hopes to make writing multi-threaded apps easier; Project Strigi will make desktop search seamless and has a DBUS interface; does meta-data search as well. Aaron also mentioned working with HCI teams to make KDE more usable to regular users - I personally like this idea. The KDE project's UI ideas have always seemed juvenile to me.
Luke Kanies talked about Puppet - a server automation tool that comes with it's own declarative language. Puppet is written in Ruby and is the result of Luke's frustration with existing tools. 'Developers do not think about manageability', feels Luke. He also added that there are many opportunities to add small pieces of functionality. Puppet has a central server, a portable resource layer and agents on the network that do the actual automation work. The custom configuration language can be versioned and changes tracked easily, since it's plain text. The author of Puppet is very interested in building an open community around this GPL'd software and said that it might take many generations to make the tool great, and he would help create those generations. Resources in Puppet could be anything - Packages, Users, Groups - just about any low level system object. Resources can have multiple low-level providers. The platform specific details are abstracted away. Clients or Agents provide the server with facts about the hosts every thirty minutes or so. Puppet uses SSL everywhere. Though it's been around for only a little while, it is already in production in a couple of places in Europe and US. Stanford Univ. is a paying client. There is no windows version yet and I could not help feeling that the lack of a GUI might be a disincentive to most admins.
This is one area in which I have lots of personal interest, and my current job also involves similar work. I chatted with Luke a bit. I think it's high time we had a good cross platform (*nix, Linux, Mac, Windows)open source tool in this area. It's time to kick the pricey vendors out - they waste our time and money, and have no clue about real requirements. Good luck to Puppet and Luke Kanies. Google videos of his talks elsewhere.
PHP man Rasmus Lerdorf gave a great talk on how to build a good community site and optimizing sites to handle high traffic; even though he used PHP as the example, I thought it was quite general. Rasmus appeared to be a bit upset with the loads of criticism levelled at PHP these days and pointed out the people making negative blog posts about PHP do not realize that Wordpress was a PHP application. I am not sure if this is a good defense of PHP, the language has real problems. But I think it's helpful to remember that for a long time and even now, it's a great way to get things done quickly. As he himself put it, PHP is for weekend warriors. Rasmus actually began his talk by hypothesizing about why geeks are so motivated to work on open source software or even do non-programming community work. He mentioned the case of some users on Yahoo Answers who answer 500-1000 per day questions some times and attributes it to oxytocin, the hormone that gets released during human interaction. Even geeks who rarely see daylight and live in their mom's basement, have the need to connect with other humans, and hence the motivation to take part in net communities. Rasmus feels that it's not what people think about you, but rather what they think about themselves that matter.
Rasmus observed that people should be able to improve a community site by catering to their self interest;he mentioned that no worthwhile community can be built without giving up ownership - if you don't give up control, you will never build a decent community of users. I personally think this works for some categories of software/communities and not for all. Dan Bernstein, the author of Qmail maintains total control over the direction of the project, Napster was Shawn Fanning mainly, and I could go on and these were projects that were so useful that the community ignored the idiosyncrasies of the owner. PHP is now mainly community driven, and became so around 1997; Rasmus' code in PHP is less than ten percent now.
He took apart a small PHP app with the help of Valgrind and showed how profiling an application can be counter intuitive. It was extremely interesting to hear this portion of the talk and I was unaware that valgrind could be put to such use; http_load was used to load test the site. He also mentioned the XML support in PHP5 including Simple XML, Yahoo'd geocoding api, and cautioned against using participation gimmicks like paid blogging. Solve one problem, a clean and intuitive UI were other tips from Ramus. Surprisingly he said that he finds programming to be boring and tedious, and his main interest is in solving problems. Slides here.
Christophe Hellwig, a linux kernel hacker tried to motivate contributions to the linux kernel. Among other things I learnt that one doesn't have to sign over copyrights when submitting code to the Linxu kernel. In contrast contributions to FSF projects need to sign over copyrights to the FSF and do some paper work as well. Christophe said that for result-oriented people there are lots of interesting problems to solve. Lots of modules need help. He explained how corporate contributors to the Kernel were often at conflict with the kernel developers due to pressures from the boss. There was a Sardarji in the audience who asked how many years it generally took to become a kernel developer - the audience laughed for no apparent reason ;-)
Andrew Cowie made an energetic presentation about the structural problems facing open source projects. He outlined at length the problems with centralized version control, build systems, bug tracking systems, and how they all combined to make collaboration between projects difficult. He made the very interesting observation about the committer/non-committer divide created by centralized version control systems. I wonder if DVCSs really solve the problem - you could have your own repository, but what if your changes are never merged into the main repository - C'mon stop pretending that there will be no 'main' repository with distributed version control systems. Besides the problem is how do you discover these repositories - DVCSs do not address the question of discovery at all.
The penultimate day of the conference saw a panel discussion by Sudhar 'Thaths' Chandrashekar; the panelists included Atul Chitnis, Kishore Bhargava, Karunkar Guntapalli, Arun Sharma, Frederick Noronha, KDE developer Sirtaj Singh Kang,and another bloke whose name I didn't get. Atul Chitnis who incidentally does not have too much by way of actual contribution to FOSS projects was predictably the most vocal about young people these days, yada, yada and the death of curiosity. He even made some vague connection between the ability to take apart devices and the ability to hack. In fact there is no such connection. Check out Karl Fogel's excellent article . Only in India do such vague assertions fly without fear of contradiction - it is no better than corporate software environments where the 'get things done' types and 'Excel gymnasts' are regarded as being more useful than good programmers. One recent college graduate finally got fed up with 'oh-young-people-these-days' rants and explained some interesting work they had done at college, that helped reduced the cost of the implementation. He explained that they customised the software extensively and the proprietary choice was very expensive. He also felt that Rs.500/- was a little high for college students. Instead of patiently explaining to a newbie, Atul lit on to this poor chap. One of the panelists was very honest and admitted that he had failed in his MCA exams, unlike lots of pretentious desi FOSS folks who are still enamoured of pedigree. The questions were all predictable, but the answers were better even though the pessimistic tone was a tad too much for me. I can't figure out how Atul arrived at the conclusion that things were going downhill - In fact there have never been so many Indian contributors as now. Atul and co seem to think that being well informed and following every single mailing list out there, and latching on to every passing tech fad is what contribution is all about. Well Atul, with my limited experience in open source projects I can tell you that working on nontrivial patches is real work - the kind of work that requires you to put down your head and do real hard work. Next time you feel like making a claim like this, please do some honest research and go through the commit mailing lists and actual dev lists. I know that you are playing to the gallery and pretending to be the sole saviour of open source in India. Just as life is what happens when you are planning, real open source contribution is what happens when non-programmers who are over the hill are talking about the glorious days of tinkering with modems. Honestly I don't know what the OSS contributions of the panelists other than Sirtaj, and Karunakar are.
It was funny that while one the very visible participants who is a Google employee did not miss an opportunity to let everyone know that he was working with the world's largest organized cult, I heard from another participant at the conference that a very respected open source contributor and O'Reilly author had resigned from Google after spending just three months there. A few days back news of another very respected oss contributor quitting Google trickled in. I know this guy and he is not the one to go around advertising his coolie status even if he works for google. Integrity and self respect count for more than grovelling before the benevolent master.
I attended this event after six years. The first time I met RMS. Nat Friedman's behaviour during his presentation was extraordinarily juvenile - he used the F-word so many times during the presentation that I got irritated. I didn't get a good impression and the many twists and turns of the GNOME project since then have only confirmed the fact that it is run by a bunch of juvenile delinquents. The recent Novell-Microsoft fiasco only serves to confirm my initial impressions.
Best comment/observation of FOSS.in 2006
When a member of the audience, strangely enough a Novell employee, asked if it was not possible to enjoy the freedom without getting into license wars, someone from the back (either Luke Kanies or Andrew Cowie) shouted:
That's a fan club - not an open source software project.
Sunday, January 01, 2006
Indian Code contribution to FLOSS
There have been some discussions about why there is so little code contribution from Indian programmers to free software/open source projects following the recently concluded FOSS.in conference. Linux Weekly news has some interesting observations to make:
Even if only 10% of those attendees were truly active in kernel development, one would expect to see a significant amount of code from Bangalore working its way into the mainline kernel. And there are some Bangalore-based kernel hackers who are active on the mailing lists and who are contributing code. But their numbers are far smaller than one would expect after seeing how many people are interested and knowledgeable in this area. India is, as one developer put it, "the world's biggest consumer of free software," but it is not a huge contributor.
The usual suspect, the much maligned (and rightly so) educational system in India:
For such a developer, the free software community is not a particularly attractive or welcoming place. A developer who contributes to a free software project may earn a strong reputation in the community, but that reputation is not appreciated by that developer's employer or co-workers, and is not helpful for his or her career. Criticism from the community - even routine criticism of a patch by people who appreciate the developer's contributions in general - can be hurtful to a career in a culture where open criticism is not the normal way of doing things. Developers who expect to have their job parameters laid out to them in detail may feel lost in a project where they are expected to find something useful to do, and push it forward themselves. And these developers, while being possibly quite skilled in what they do, often have no real passion for programming, and leave it all behind when they leave the office each day.
LWN's take on the issue is mostly correct. I am stunned at how close the article has come close to diagnosing the problem, and if all of it is from a one week visit to Bangalore to attend foss.in, it's the all the more impressive. But there is very little input from developers who have actually made headway in sending patches and becoming committers in FLOSS projects and that makes the article a little bit shallow, as also it's failure to consider socio-economic reasons. Regarding the latter, I think the best explanation comes from Thaths from mailing list thread:
Based on a sampling of my White geek friends, I would say they are predominantly children of white collar baby boomers. Many of them were exposed to computers their dads brought home from work at an early age. I think similar conditions exist in India today for the kids of today.
I came to the same conclusion a couple of years back and have noticed that most of the American hackers whom I have read about, or have worked with, have had very early exposure to computers and programming and their comfort level is consequently very high. I am starting to notice that a bit with the generation born in the 80's and 90's in India as well. They tinker around with hardware with relative ease, play more games, and seem to know their way around computers much better than their parents, who incidentally slogged it out to create this improved environment for them. However I think even the socio-economic reason is only partial, and no one really knows. There is a real need to put on a web page the Indian code contributions and collect the experiences of Indian hackers.
Even if only 10% of those attendees were truly active in kernel development, one would expect to see a significant amount of code from Bangalore working its way into the mainline kernel. And there are some Bangalore-based kernel hackers who are active on the mailing lists and who are contributing code. But their numbers are far smaller than one would expect after seeing how many people are interested and knowledgeable in this area. India is, as one developer put it, "the world's biggest consumer of free software," but it is not a huge contributor.
The usual suspect, the much maligned (and rightly so) educational system in India:
By several accounts, the problem starts with the university system. The Indian universities are strongly oriented toward the creation of employable graduates in large numbers; a number of FOSS.IN attendees described them as "assembly line" operations. There is a strong emphasis on passing tests and getting through the system on schedule, and, it seems, little interest in encouraging creativity and curiosity in the students. The universities were described as a conformist environment with little love of those who have their own ideas of how things should be done. The end result, as expressed to your editor, is that most students have had any love of hacking beaten out of them by the time they graduate.
The fact that the universities are, for the most part, hostile to Linux and free software does not help either.
The problems with the typical corporate developer in India are outlined:
Neeti's talk described Indian developers as needing to have their jobs laid out to them in great detail. They want to know where their boundaries are, and are uncomfortable if left to determine their own priorities and approaches. Your editor's initial reaction was that this claim sounded like classic talk from a pointy-haired boss who does not trust his employees to make decisions. Subsequent discussions backed up Neeti's claims, however. A few Indians told me that Indian employees require a high degree of supervision; perhaps that is why the pizza stand at the site required two-levels of necktie-wearing bosses who apparently did little to actually get pizza into the hands of conference attendees. It is not that Indians lack the intelligence to function without a boss breathing down their neck - that is clearly not the case - but all of their training tells them to work in that way.
So if one were to construct a stereotypical picture of an Indian software developer, it would depict a person who sees programming very much as a job, and not as an activity which can be interesting or rewarding in its own right. This developer is most interested in getting - and keeping - a stable job in a country where an engineering career can be a ticket to a relatively comfortable middle-class existence. Keeping that job requires keeping management - and coworkers - happy, and not rocking the boat.
For such a developer, the free software community is not a particularly attractive or welcoming place. A developer who contributes to a free software project may earn a strong reputation in the community, but that reputation is not appreciated by that developer's employer or co-workers, and is not helpful for his or her career. Criticism from the community - even routine criticism of a patch by people who appreciate the developer's contributions in general - can be hurtful to a career in a culture where open criticism is not the normal way of doing things. Developers who expect to have their job parameters laid out to them in detail may feel lost in a project where they are expected to find something useful to do, and push it forward themselves. And these developers, while being possibly quite skilled in what they do, often have no real passion for programming, and leave it all behind when they leave the office each day.
LWN's take on the issue is mostly correct. I am stunned at how close the article has come close to diagnosing the problem, and if all of it is from a one week visit to Bangalore to attend foss.in, it's the all the more impressive. But there is very little input from developers who have actually made headway in sending patches and becoming committers in FLOSS projects and that makes the article a little bit shallow, as also it's failure to consider socio-economic reasons. Regarding the latter, I think the best explanation comes from Thaths from mailing list thread:
Based on a sampling of my White geek friends, I would say they are predominantly children of white collar baby boomers. Many of them were exposed to computers their dads brought home from work at an early age. I think similar conditions exist in India today for the kids of today.
I came to the same conclusion a couple of years back and have noticed that most of the American hackers whom I have read about, or have worked with, have had very early exposure to computers and programming and their comfort level is consequently very high. I am starting to notice that a bit with the generation born in the 80's and 90's in India as well. They tinker around with hardware with relative ease, play more games, and seem to know their way around computers much better than their parents, who incidentally slogged it out to create this improved environment for them. However I think even the socio-economic reason is only partial, and no one really knows. There is a real need to put on a web page the Indian code contributions and collect the experiences of Indian hackers.
Marc Canter on Chandler
Marc Canter says about Chandler:
Three years later and at least $5M Chandler hits version .6. Man oh man what we could do with that sort of money!
Three years later and at least $5M Chandler hits version .6. Man oh man what we could do with that sort of money!
Tuesday, October 11, 2005
The Best Program according to Don Knuth
The Nerd TV interview with Andy Hertzfeld contains another interesting bit. According to Don Knuth, MacPaint was possibly the best program ever written:
I have another anecdote that has to do with Open Source and the Macintosh. In January, 2004, the Computer History Museum had a little presentation about the marketing of the Macintosh with all the original Macintosh marketing team up on the stage. And a large percentage of the rest of the Mac team in the audience. They had a Q&A toward the end and an older guy got up and said he thought MacPaint was probably the best program ever written. Was it possible for him to see the source code? It turns out the person asking the question was Don Knuth - another one of my heroes. He came up to me afterwards.
I have another anecdote that has to do with Open Source and the Macintosh. In January, 2004, the Computer History Museum had a little presentation about the marketing of the Macintosh with all the original Macintosh marketing team up on the stage. And a large percentage of the rest of the Mac team in the audience. They had a Q&A toward the end and an older guy got up and said he thought MacPaint was probably the best program ever written. Was it possible for him to see the source code? It turns out the person asking the question was Don Knuth - another one of my heroes. He came up to me afterwards.
Andy Hertzfeld on GNOME
I remember buying the Eazel T-shirt to show my appreciation for the folks who had shown the earliest signs of good taste in the free desktop world, back when I thought that the Linux desktop was going to make a huge impact. Raph's SVG work which resulted in the first SVG icons on the desktop were pretty cool, Nautilus generally seemed to be a notch above the rest as far as polish and attention to detail were concerned. Then came the crash and Eazel dissapeared. In an interview with Nerd TV Andy Hertzfeld explains what they set out to do and how things have actually turned out with GNOME.
We were just building on top of Gnome. Our goal was to make Linux easier to use and part of that is work on the GUI side of things and part of that was work on system management. We evaluated the alternatives and ended up picking Gnome. We built something called Nautilus, a new file manager. I never thought of it as a file manager, I'd call it a graphical shell. We got about halfway done what we wanted to. It's just disappointing we never got to take it as far as we wanted.
It is still used. If you install Gnome on your machine you'll be installing Nautilus. It has drifted away… it didn't live up to its potential. It still has some good stuff in it, but he people who took it over did not have much forward thinking vision. They ended up, "when all else fails just copy Windows." So it has kind of drifted into that modality.
We were just building on top of Gnome. Our goal was to make Linux easier to use and part of that is work on the GUI side of things and part of that was work on system management. We evaluated the alternatives and ended up picking Gnome. We built something called Nautilus, a new file manager. I never thought of it as a file manager, I'd call it a graphical shell. We got about halfway done what we wanted to. It's just disappointing we never got to take it as far as we wanted.
It is still used. If you install Gnome on your machine you'll be installing Nautilus. It has drifted away… it didn't live up to its potential. It still has some good stuff in it, but he people who took it over did not have much forward thinking vision. They ended up, "when all else fails just copy Windows." So it has kind of drifted into that modality.
The 37 Signals Way
I remember 37 signals vaguely as the company that the Ruby on Rails guy works for and they seem to generate quite a buzz from time to time with bloggers, for reasons that I have never cared to investigate. That is until I listened to a podcast - "Lessons learned while building BaseCamp" by Jason Fried, the founder of 37 signals recorded at the O'Reilly emerging technology conference earlier this year. What blew me away was the refreshing approach to software development these guys have and how cogently it was articulated. While Jason Fried himself admits inspiration from Agile programming, I think it deserves attention on it's own merits, and for being able to set out a bunch of actionable items. Putting a new spin something that many people might know or might be obvious to others is not without merits, there is always value in re-packaging good 'old' ideas.
The semi-transcription/notes from the talk:
The talk revolves around four main themes :
Mass could be people, high overheads, long term contracts etc. It's difficult to change with more mass, and being able to change is extremely important for any product development process. Find the right people -we look for people with certain level of skills. But, beyond a certain point it is not about skills alone, it's not about how many years experience you have with PhotoShop, it's not about bullet points. You need to work with people that are positive most of the time. In a small team of four one disgruntled person can bring down the whole team. Morale is incredibly important when building a product.You have to have people who are excited about what they are doing and are positive about what they are doing. Jason advocates hiring people who are well rounded, and can do multiple things. Don't hire someone who is just a programmer, or an information architect, or a designer, or a marketing person, especially on small teams. Hire people who can learn quickly new things (like AJAX) and apply it quickly.Hire Trustworthy people - trustworthy does not mean people who lock the doors at nights and remember to turn off the lights. It's very important that you don't have to cleanup after that person. The person you want to hire has to be a good writer - A programmer, or a designer, or a marketing person all have to be good writers, because of how people communicate with each other to each other these days - they don't talk that much to each other these days, they IM/email each other all the time. You have to find people who can communicate by writing. Jason's bottom line for hiring people - I will take someone who is happy and average, rather than someone who is a Guru, who is the best in the business, but is disgruntled and frustrated. Such people will bring down the team in the long run.
Constraints
Once you have put in the team in place, you have put in the constraints. Most companies instead of embracing constraints start thinking, we don't have enough money - let's raise more money, we don't have enough people - let's hire more people, and so on. Don't do that! Look at your problems and come up with creative solutions. Embrace those problems, and don't push them under the rug. Some of the constraints that 37 signals had to deal with when building BaseCamp:-
Prior commitments: 37 signals was still a web design shop and initially BaseCamp was about 20% of the time spent. David Hanson the only programmer on the project was still in school and was committed to some of his own free lance work as well. We didn't put everything down and say 'let's focus on BaseCamp'. We made sure that the time spent on base camp is valuable time. The time difference between Chicago and Copenhagen (where Mr.Ruby on Rails lives) is seven hours -On paper it looks like disaster - we decided it was a good thing, you communicate mostly through IM and email. When you talk through IM, you have to be really specific. You chunk things into small pieces and you move on. We are self-funded. Spending money should be the hardest thing that you ever do. Make sure that every penny that you spend is a hard penny to spend. That will ensure that you end up making much better decisions and you spend money in better places. And finally having a small team - we had a team of three people, one programmer, and two designers. People in smaller teams have more power, you can't hide behind twenty other people.Everyone has to do something. It ensures that everyone cares about the product. These were constraints that were put upon us and there are ones that you can put on yourself. Building less software - that's the constraint that we decided to impose on ourselves. It's not about building feature-by-feature, but less features intentionally. When you have less software, you lower the cost of change.
Managing the opportunity for things to go wrong is the key thing. Less support is required. Technical support will kick your ass - It will destroy you. We don't advocate hiring a bunch of people to do technical support. You want to build something that is manageable. Outsourcing tech support is a bad thing. Finally, you also want to encourage human solutions. Less software, less features means less rigidity which means that people have to come up with their own way of dealing with the product. We all hate the paper clip guy in the corner - you should give people enough to solve their problems in their own way and then get out of the way.Ta-da list is an example - a really simple to-do manager.You look at the patterns in the users behavior and maybe there is an opportunity there to improve the product based on how people use it, based on real product use, not fake paper mock-ups etc. Look for usage patterns in a real environment and add a feature if it turns out to be necessary or if people ask for it. It's always better to start with less than more. The idea of building less software leads to the next idea - how do you do releases - You release half-a-product, not a half-assed product. There should be no scope creep, only scope reduction.Everyone is doing a beta these days and that's ridiculous, Everything is a work in progress and continue to improve it , have confidence in your product.
How to do scope reduction
Say No, by default, to all new features. You want to listen to the people who are using it. Your customer base will tell when you need to add something. If you get ten requests, then it's a "maybe", if you get twenty then it is stronger (demand), thirty requests - then probably it is something you need to add. Listen to your product, and listen to your customers. Say No by default, see where the gaps are, observe the usage patterns before deciding to add a feature. You will build a better product based on real data.
You want to ignore details early on - you can spend a week on something that doesn't really matter. A lot of small things that you think matter don't really matter. Take care of the big picture first. Fill in the details close to the release. If you have time before the release make existing stuff better instead of adding new features. The idea that every minute should be spent on a new feature is a bad idea. Focus on what you have - make it rock solid, make it bullet proof, make it great. There is always room to add stuff later.
All decisions are temporary. You should be able to change all decisions that you make, in theory - realistically you should be able to change almost all. You can change decisions if you have less mass. You want to make decisions that you can change, every decision should be temporary. You should should take decisions just in time - You should make decisions when you have real information to take a decision.
Make mistakes in the open, admit it to customers and fix it quickly. Jason admits that this is not applicable to every kind of project. It applies to web based software, especially the UI portion. Make small changes, iterate often.
Getting Real
We don't do functional specs, we don't do diagrams, we don't spend a lot of time doing use cases, we don't spend a lot of time doing things the customer will never see or never use. We want to be in front of the customer experience close to 100% of the time.If it takes four months to make a product I want to spend 3.9 months in front of the actual product.I don't want to slap user experience at the end. That's what happens when you are talking about these functional specification documents - functional specs are political documents.They are documents used to blame one another - it was supposed to work this way,it doesn't etc.It's also an illusion of agreement. Someone will say inevitably - "that's not what I intended".Well you signed off on it and politics starts. The design is the functional specification - programming needs to follow the user interface. When you are in front of your interface all the time, it is almost like your version 1.0 is someone else's version 2.0, because you are constantly tweaking real user experience that is not based on paper prototypes - people always have something to offer when they see the real thing. You are postponing those inputs by writing functional specs. Build simple things for all users including expert users.
Managing Debt
Manager your debt - If you have earned debt through some ugly hacks, pay it off as soon as possible by setting the code right. Don't get stuck in hardware or software that you can't change.
Making most decisions just in time
Japaneses factories that have just what they need, use new or better parts. Just in Time systems allow people to make better products. Scalability is a myth. If you are a yahoo or a google, then you really need to worry about it. If something is slow for 2 weeks, it is slow for 2 weeks. It's OK to make mistakes in public. BaseCamp was running on a single server and was being used by thousands and thousands of users, till about three months back. Then we switched to 2 boxes and we are in the process of switching to eight. If you invest too much money and effort up front setting up a cluster and you don't need that for 12 months you have wasted an awful lot of money and you could have spent the time on something more valuable. Admin interfaces are another - don't spend time on writing stuff that will generate extensive reports and what not. Actual customer experience first. BaseCamp is free for 30 days. We couldn't bill anyone in the beginning;figured it out after a while. Make decisions when you have the real information, not before.
Feeling the Hurt
The people who build the product should do the tech support on the product for as long as possible. Jason still doing tech support and he owns the company. I want to know what's right, what's wrong. 37 Signals get only forty emails a day - simple product that is well designed. Don't want to get the same emails over and over again. Chefs have to be waiters in certain restaurants.
Publicity Amplifiers
Here are some publicity amplifiers - for getting the word out about your product. With no marketing budget,no PR budget,no money, how to get the word out ? Lace the product with feature food - features that users use and spit out small little features and people tell others about it. Small little features that users eat and spit out all over the place. In BaseCamp you can subscribe to your feature milestones via Ical. We talked to mac sites about it - this news blurb generated tons of traffic and tons of sales. Same feature is available via RSS as well - the blog world loves RSS. The Yellow Fade technique generated about 325 references on google. This small technique generated more buzz than the product itself. Hold back some features, and do it 30 days after launch. It will show that there is momentum happening. people will really love that and it shows your commitment to the product - You didn't build a product only to move on to something else.
Transparency and Trust
If you make a mistake, tell people.We made a mistake and had to be manually in front of our servers for forty hours. No data was lost and we had to re-start the server constantly. We could have hidden that;but, we posted a long explanation - lot of people trust us now.
Blogging
Blog as much as possible about your product. Google will send lot of traffic your way.
Question from the audience:
How do you work with David Hanson (who is in Denmark) ?
Jason: We use BaseCamp to do 8-10 hr iterations in order of priority - we don't worry about what will happen eight months down the road.
Looks like Ruby on Rails is not the only thing to come out a small software startup with less than half a dozen employees.
The semi-transcription/notes from the talk:
The talk revolves around four main themes :
- Reducing Mass
- Embrace Constraints
- Getting Real
- Managing Debt
Mass could be people, high overheads, long term contracts etc. It's difficult to change with more mass, and being able to change is extremely important for any product development process. Find the right people -we look for people with certain level of skills. But, beyond a certain point it is not about skills alone, it's not about how many years experience you have with PhotoShop, it's not about bullet points. You need to work with people that are positive most of the time. In a small team of four one disgruntled person can bring down the whole team. Morale is incredibly important when building a product.You have to have people who are excited about what they are doing and are positive about what they are doing. Jason advocates hiring people who are well rounded, and can do multiple things. Don't hire someone who is just a programmer, or an information architect, or a designer, or a marketing person, especially on small teams. Hire people who can learn quickly new things (like AJAX) and apply it quickly.Hire Trustworthy people - trustworthy does not mean people who lock the doors at nights and remember to turn off the lights. It's very important that you don't have to cleanup after that person. The person you want to hire has to be a good writer - A programmer, or a designer, or a marketing person all have to be good writers, because of how people communicate with each other to each other these days - they don't talk that much to each other these days, they IM/email each other all the time. You have to find people who can communicate by writing. Jason's bottom line for hiring people - I will take someone who is happy and average, rather than someone who is a Guru, who is the best in the business, but is disgruntled and frustrated. Such people will bring down the team in the long run.
Constraints
Once you have put in the team in place, you have put in the constraints. Most companies instead of embracing constraints start thinking, we don't have enough money - let's raise more money, we don't have enough people - let's hire more people, and so on. Don't do that! Look at your problems and come up with creative solutions. Embrace those problems, and don't push them under the rug. Some of the constraints that 37 signals had to deal with when building BaseCamp:-
Prior commitments: 37 signals was still a web design shop and initially BaseCamp was about 20% of the time spent. David Hanson the only programmer on the project was still in school and was committed to some of his own free lance work as well. We didn't put everything down and say 'let's focus on BaseCamp'. We made sure that the time spent on base camp is valuable time. The time difference between Chicago and Copenhagen (where Mr.Ruby on Rails lives) is seven hours -On paper it looks like disaster - we decided it was a good thing, you communicate mostly through IM and email. When you talk through IM, you have to be really specific. You chunk things into small pieces and you move on. We are self-funded. Spending money should be the hardest thing that you ever do. Make sure that every penny that you spend is a hard penny to spend. That will ensure that you end up making much better decisions and you spend money in better places. And finally having a small team - we had a team of three people, one programmer, and two designers. People in smaller teams have more power, you can't hide behind twenty other people.Everyone has to do something. It ensures that everyone cares about the product. These were constraints that were put upon us and there are ones that you can put on yourself. Building less software - that's the constraint that we decided to impose on ourselves. It's not about building feature-by-feature, but less features intentionally. When you have less software, you lower the cost of change.
Managing the opportunity for things to go wrong is the key thing. Less support is required. Technical support will kick your ass - It will destroy you. We don't advocate hiring a bunch of people to do technical support. You want to build something that is manageable. Outsourcing tech support is a bad thing. Finally, you also want to encourage human solutions. Less software, less features means less rigidity which means that people have to come up with their own way of dealing with the product. We all hate the paper clip guy in the corner - you should give people enough to solve their problems in their own way and then get out of the way.Ta-da list is an example - a really simple to-do manager.You look at the patterns in the users behavior and maybe there is an opportunity there to improve the product based on how people use it, based on real product use, not fake paper mock-ups etc. Look for usage patterns in a real environment and add a feature if it turns out to be necessary or if people ask for it. It's always better to start with less than more. The idea of building less software leads to the next idea - how do you do releases - You release half-a-product, not a half-assed product. There should be no scope creep, only scope reduction.Everyone is doing a beta these days and that's ridiculous, Everything is a work in progress and continue to improve it , have confidence in your product.
How to do scope reduction
Say No, by default, to all new features. You want to listen to the people who are using it. Your customer base will tell when you need to add something. If you get ten requests, then it's a "maybe", if you get twenty then it is stronger (demand), thirty requests - then probably it is something you need to add. Listen to your product, and listen to your customers. Say No by default, see where the gaps are, observe the usage patterns before deciding to add a feature. You will build a better product based on real data.
You want to ignore details early on - you can spend a week on something that doesn't really matter. A lot of small things that you think matter don't really matter. Take care of the big picture first. Fill in the details close to the release. If you have time before the release make existing stuff better instead of adding new features. The idea that every minute should be spent on a new feature is a bad idea. Focus on what you have - make it rock solid, make it bullet proof, make it great. There is always room to add stuff later.
All decisions are temporary. You should be able to change all decisions that you make, in theory - realistically you should be able to change almost all. You can change decisions if you have less mass. You want to make decisions that you can change, every decision should be temporary. You should should take decisions just in time - You should make decisions when you have real information to take a decision.
Make mistakes in the open, admit it to customers and fix it quickly. Jason admits that this is not applicable to every kind of project. It applies to web based software, especially the UI portion. Make small changes, iterate often.
Getting Real
We don't do functional specs, we don't do diagrams, we don't spend a lot of time doing use cases, we don't spend a lot of time doing things the customer will never see or never use. We want to be in front of the customer experience close to 100% of the time.If it takes four months to make a product I want to spend 3.9 months in front of the actual product.I don't want to slap user experience at the end. That's what happens when you are talking about these functional specification documents - functional specs are political documents.They are documents used to blame one another - it was supposed to work this way,it doesn't etc.It's also an illusion of agreement. Someone will say inevitably - "that's not what I intended".Well you signed off on it and politics starts. The design is the functional specification - programming needs to follow the user interface. When you are in front of your interface all the time, it is almost like your version 1.0 is someone else's version 2.0, because you are constantly tweaking real user experience that is not based on paper prototypes - people always have something to offer when they see the real thing. You are postponing those inputs by writing functional specs. Build simple things for all users including expert users.
Managing Debt
Manager your debt - If you have earned debt through some ugly hacks, pay it off as soon as possible by setting the code right. Don't get stuck in hardware or software that you can't change.
Making most decisions just in time
Japaneses factories that have just what they need, use new or better parts. Just in Time systems allow people to make better products. Scalability is a myth. If you are a yahoo or a google, then you really need to worry about it. If something is slow for 2 weeks, it is slow for 2 weeks. It's OK to make mistakes in public. BaseCamp was running on a single server and was being used by thousands and thousands of users, till about three months back. Then we switched to 2 boxes and we are in the process of switching to eight. If you invest too much money and effort up front setting up a cluster and you don't need that for 12 months you have wasted an awful lot of money and you could have spent the time on something more valuable. Admin interfaces are another - don't spend time on writing stuff that will generate extensive reports and what not. Actual customer experience first. BaseCamp is free for 30 days. We couldn't bill anyone in the beginning;figured it out after a while. Make decisions when you have the real information, not before.
Feeling the Hurt
The people who build the product should do the tech support on the product for as long as possible. Jason still doing tech support and he owns the company. I want to know what's right, what's wrong. 37 Signals get only forty emails a day - simple product that is well designed. Don't want to get the same emails over and over again. Chefs have to be waiters in certain restaurants.
Publicity Amplifiers
Here are some publicity amplifiers - for getting the word out about your product. With no marketing budget,no PR budget,no money, how to get the word out ? Lace the product with feature food - features that users use and spit out small little features and people tell others about it. Small little features that users eat and spit out all over the place. In BaseCamp you can subscribe to your feature milestones via Ical. We talked to mac sites about it - this news blurb generated tons of traffic and tons of sales. Same feature is available via RSS as well - the blog world loves RSS. The Yellow Fade technique generated about 325 references on google. This small technique generated more buzz than the product itself. Hold back some features, and do it 30 days after launch. It will show that there is momentum happening. people will really love that and it shows your commitment to the product - You didn't build a product only to move on to something else.
Transparency and Trust
If you make a mistake, tell people.We made a mistake and had to be manually in front of our servers for forty hours. No data was lost and we had to re-start the server constantly. We could have hidden that;but, we posted a long explanation - lot of people trust us now.
Blogging
Blog as much as possible about your product. Google will send lot of traffic your way.
Question from the audience:
How do you work with David Hanson (who is in Denmark) ?
Jason: We use BaseCamp to do 8-10 hr iterations in order of priority - we don't worry about what will happen eight months down the road.
Looks like Ruby on Rails is not the only thing to come out a small software startup with less than half a dozen employees.
Friday, August 26, 2005
Bruce Eckel on Consulting
Great post from Bruce Eckel on consulting. His definition:
"I think consulting is when you have some kind of special expertise -- come by through hard struggle and learning -- that you transfer to a group of people, in a relatively short period of time, and in a way that is unique for that group. I also think that consulting involves addressing particular issues faced by that group. "
He also addresses a very common industry phenomenon:
"It's tempting to make the analogy to a ponzi scheme, but it's not that bad. Both forms of a consulting firm (first-class consultants only vs. high-tech body shop) have value. The problem is the temptation to present the second form as the first, in order to apply the same high fees of the first-class consultant to each additional body added to the shop. The goal of the company becomes "how do we transfer the aura of authority from the high-image consultant(s) to anyone who works for us, so that we can charge the highest fees possible?" Or to simplify, at some point the bean-counter mentality takes over and the mission statement of the company goes from "how do we provide the greatest value to the customer?" to "how do we charge the highest fees possible?" (You can argue that this is the fundamental shift that any publicly-held company goes through. After all, a public company is legally beholden to maximize shareholder value, so how could it be otherwise?)"
I have noticed this myself, when I was working in Boston during the tail portion of the dot-com years. I was chatting to an Indian body shopper who was a sub-contractor for Andersen Consulting; While Andersen made $450 plus per hour, this sub-contractor got about $70 - You can imagine how much would have trickled down ultimately into the pockets of the programmers, who were paraded as Andersen consultants to clients.
Another nugget from the article:
But advice that appears to be rational does not necessarily solve real design problems, as we saw in the "structured design" movement of the 70's. This seems to happen over and over in our industry -- it's easy to focus on one aspect that seems to be "the solution" and miss the big picture, and to produce unintended consequences that eliminate the benefit of what may seem so clear in isolation.
"I think consulting is when you have some kind of special expertise -- come by through hard struggle and learning -- that you transfer to a group of people, in a relatively short period of time, and in a way that is unique for that group. I also think that consulting involves addressing particular issues faced by that group. "
He also addresses a very common industry phenomenon:
"It's tempting to make the analogy to a ponzi scheme, but it's not that bad. Both forms of a consulting firm (first-class consultants only vs. high-tech body shop) have value. The problem is the temptation to present the second form as the first, in order to apply the same high fees of the first-class consultant to each additional body added to the shop. The goal of the company becomes "how do we transfer the aura of authority from the high-image consultant(s) to anyone who works for us, so that we can charge the highest fees possible?" Or to simplify, at some point the bean-counter mentality takes over and the mission statement of the company goes from "how do we provide the greatest value to the customer?" to "how do we charge the highest fees possible?" (You can argue that this is the fundamental shift that any publicly-held company goes through. After all, a public company is legally beholden to maximize shareholder value, so how could it be otherwise?)"
I have noticed this myself, when I was working in Boston during the tail portion of the dot-com years. I was chatting to an Indian body shopper who was a sub-contractor for Andersen Consulting; While Andersen made $450 plus per hour, this sub-contractor got about $70 - You can imagine how much would have trickled down ultimately into the pockets of the programmers, who were paraded as Andersen consultants to clients.
Another nugget from the article:
But advice that appears to be rational does not necessarily solve real design problems, as we saw in the "structured design" movement of the 70's. This seems to happen over and over in our industry -- it's easy to focus on one aspect that seems to be "the solution" and miss the big picture, and to produce unintended consequences that eliminate the benefit of what may seem so clear in isolation.
Slash Wisdom
From an "Ask Slashdot" posting titled "Uneducated IT Managers, and How to Deal ?", I found this gem posted by Pope Benedict XVI ;-) :
"It's just about impossible to find a job working for someone whom you respect. "
To me, that about summarizes the crux of the problem with working in the IT industry. But strangely this nugget of wisdom has been marked as funny, instead of insightful, or some such. Setting the filter at four and reading reveals more insightful comments. Slashdot, for it's flaws, sometimes throws up insights that knock out your breath momentarily, or better still has the reader in splits - Some of the trolls are outrageously funny without being crude or offensive.
Wisdom of the commons, or shall we say, Slash Wisdom !
"It's just about impossible to find a job working for someone whom you respect. "
To me, that about summarizes the crux of the problem with working in the IT industry. But strangely this nugget of wisdom has been marked as funny, instead of insightful, or some such. Setting the filter at four and reading reveals more insightful comments. Slashdot, for it's flaws, sometimes throws up insights that knock out your breath momentarily, or better still has the reader in splits - Some of the trolls are outrageously funny without being crude or offensive.
Wisdom of the commons, or shall we say, Slash Wisdom !
Monday, August 08, 2005
Joel on Software Writing
A few interesting points ( mostly paraphrased) from a forty minute interview with Joel Spolsky on software writing:
- Most software writing is awful. There is a high correlation between successful software and writing.
- Programmers are used to writing code for compilers; compilers don't care about order. Humans comprehend stories - can't give them a bunch of definitions and tie it together. Most programmers, who write for compilers , don't know how to write for human brains.
- There is an awful lot of writing there about technology, even internal functional specifications that just doesn't get read, because regular people don't understand it.
- At Fog Creek software (Joel's company) they ask for writing samples, not necessarily technical stuff from all candidates. Because we (Fog Creek) really do find that the ability to communicate clearly and effectively, is the crucial difference between an acceptable programmer and a great programmer that can work on teams, and get their ideas understood by the world.
- Once you make software that mediates between humans, then the design decisions in the software have anthropological implications. In Joel's latest book, Danah Boyd talks about some websites that require a strict social relations - such requirements are pathological, not normal. People are actually creating software and websites that require you to conform to a set of behaviors that in the real world would get you incarcerated.
- Paul Graham's assertions about great hackers - they use open source, they don't use Java etc flies in the face of facts. But his essays are beautifully written. Joel said his hands start shaking whenever there is a new post on Paul Graham's site.
- Phil Windley asks an interesting question about whether it is opninions that make software writing interesting. Joel says that's where the value is being added (in opinions) and goes on to add that with great respect to the American journalistic tradition of objectivity, the only thing it results in is lot of cnet's news.com, that is very bland, that reproduces a lot of press releases , or they get someone from Sun to say something about Microsoft's announcement.
- According to Joel, the number one rule of technical writing is, "Show, don't tell". Technical writers violate that rule so often. So much technical writing reads like mathematical proof - all the information is here, if you can decipher it, you can understand it. That kind of stuff doesn't get read unless it has to get read. The stuff that people read are the ones with stories in it. Human beings are story tellers. We have been telling stories around camp fires for hundreds of thousands of years - that's what out brains are good at and that's we enjoy, that's why TV is popular and that's how messages get into our head.
- When people started using PC's there was lot of piracy, lot of the books were just shovelware. Companies were trying to make the books as fat as possible. At one time software could not be used without books. It was not until the GUI generation and much later, that software companies realized that they don't have to produce the manuals at all. These days some of the best books are the ones written to explain the domain in which software is used. Quick Books' manuals, for example, teach you a lot about the basics of book keeping.
- Amazon ratings are not accurate and are not done by professional people ( I personally disagree with this sweeping characterization). Joel said that there were folks with axes to grind who wrote reviews on Amazon and gave the example of a Microsoft techie, whose book got bad ratings, because he used to be aggressive in some news groups.
- If you are a computer programmer, put yourself in another person's mind - what do they know at any given time ? Try to answer the questions in their minds in the order in which they are having it. If you write a document the reader first with what the document is about, and start answering the questions that they might have in their minds.
- Some of the great functional specs are written like a story.
- Blogging is a good way to exercise the writing muscle. Joel noted that he had gotten a bit over defensive in his writing of late.
Saturday, August 06, 2005
Friday, July 29, 2005
Miguel de Icaza on on J2EE domination
Miguel explains in an interview why J2EE has such a stranglehold in the application server market:
There is a segmentation of the application server space, which is the high-end market, the mid-market, and the low market. The low market is anything that costs less than $200,000 to develop and deploy. It includes every technology that you’ve heard of. Then you’ve got the high market, which is any project where the cost of deployment is over two million dollars, and in that market J2EE is firmly entrenched. There is no other technology considered today in that application space. In part that’s because people need to have multiple vendors providing the same solutions, so they like the fact that there’s BEA and there’s IBM and there’s Sun. There’s different J2EE providers. There’s also different hardware providers. So that market is very hard for Microsoft to penetrate."That leaves the mid-market, which today is about 50% Java, with the other 50% made up of ASP.NET, and a couple of other proprietary frameworks."
Sunday, July 24, 2005
Adam Bosworth on a new data model for the web
Following the excellent presentation by Michael Tiemann at the MySQL user's conference organized by O'Reilly media, itconversations has posted another great presentation - this time by Adam Bosworth at the same conference. The talk titled "Database requirements in the age of scalable services" and Adam Bosworth explains his vision to make data access on the web simple and standards based, and why he thinks RSS 2.0/Atom are the beginning of the unfolding of that vision.
The presentation starts with a joke about 'Microsoft Project' - Bosworth says he is amused to hear about people using Microsoft Project and how it is Microsoft's secret weapon to stop everyone else from competing. Some of the points I noted down while listening to the podcast.
The presentation starts with a joke about 'Microsoft Project' - Bosworth says he is amused to hear about people using Microsoft Project and how it is Microsoft's secret weapon to stop everyone else from competing. Some of the points I noted down while listening to the podcast.
- Everything looked irrelevant to AB after he got excited about the web.
- How did the web happen ? Tim Berners Lee hit a perfect storm of productivity and using HTML, HTTP any 'P' (Perl, Python, PHP) programmer could generate content. Simple was huge and everybody could play with HTML and HTTP. HTML is sloppy - everything is rendered
without a complaint. It takes a licking and keeps on ticking. You don't have to be the high priest of syntax. But that's not the case with XHTML - and that's not a good user experience. Web pages could read/edited on all operating systems - Databases are not good at partitioning - partitioning of data is very important to scalability. But the web does a good job of this, by distributing data. The web is also good at caching - At google they
observed 120,000 hits per second on a certain blog and the only reason the infrastructure didn't melt down was due to the amount of caching done on the web by proxies, front ends and google's own front end. Statelessness - the coarse grained interaction is also another reason for the scalability of the web. Clients talk to servers in terms of chunks of data: go to data when you are ready, not
continuously. - Google combines lot of simple minded techniques with brute force to deliver. Google has lots of Ph.D's and everyone is a General Patton driving tanks. Take the spell check that google does when you type in an incorrect (or sometimes correct) search string - it's based on the very simple technique of tracking failed searches and what users type in after a failed search. This kind of brute force enables google togo through petabytes of data in seconds.
- Once you start to search, this whole business of putting things in folders begins to diminish in importance. It's very hard with folders to remember where you put what. Folders are not efficient, searchesare.
- The Vision is to take the database and do the same thing for the web (as was done for content). Can we take all the info on the web andmake it easily findable ? Now you get content, not information.
- Need something that scales massively and linearly. Originally thought that we would do it using XML. We created a tower of Babel (with XML) - websites need to support only one grammar with HTML. A working group took four years to come up with a spec for a XML Query standard. It's
better to spend six months and learn the rest from customers. The query standard was not simple like the web - the schemas were very complicated. AB also found the WS specs to be very complicated. Why did this happen ? The companies that came up with these standards were big and were trying to protect themselves. They were people at companies like IBM and MS. Frankly, they were trying to make it deliberately hard - AB apparently is a technical advisor to MySQL and had some advice for them as well: Basically MySQL is trying to be Oracle by adding support for procedures, triggers, and views. All of this is about centralizing processing logic in the database. To be blunt centralizing processing logic in the database is a bad idea - doesn't scale. Centralizing logic doesn't give you scale. Advice to MySQL - don't do something because you want to be Oracle. Because Oracle isn't big enough andcan't deliver on billions of queries.
- We need an Open model for data. What's not open today is how you talk to a database - the actual wire format. There is nothing like HTML/HTTP for data. This is a very 20th century way of thinking. Open up wire formats to serve any kind of information - this will bring enormous changes to computing centered around data. Need open standards for different types of items with one single grammar. It will have to be sloppy. Open up and democratize the way data isserved.
- Big believer in stupidity - virtues of dumbness.
- We are actually starting to realize this vision - RSS2.0/Atom are going to be for data what HTML was for content. They are going to be the Lingua Franca of consuming data. Surprisingly simple and sloppy. These guys got the web and that's why it is catching on like wild fire. Atom was formed by a consortium of bloggers and the two formats areisomorphic.
- Data queries have to be such that they don't need data spread across machines - if a query uses data from four machines, it isn't going to work very well. Queries need to run at an item level - it's not
technically as complex as sql - AB made it emphatically clear that he was not talking about the semantic web and called RDF an empirical failure. RSS 1.0 had an RDF grammar, RSS 2.0 doesn't have an RDF based grammar. Ordinary programmers do not understand how to model something as arcs, nodes,and graphs
Saturday, July 23, 2005
Michael Tiemann on Open Source
The peerless ITConversations.com features a talk by Michael Tiemann of RedHat, formerly of Cygnus and the guy who first wrote the GNU C++ compiler. Some interesting points from the talk:
Update: Looks like Michael Tiemann gave a similar talk (pdf) at the Redhat summit.
- In 1995, when MySql started (this was a talk at the 10th anniversary MySql conference), Tiemann had already turned over the reins of the company to a president at Cygnus; the company was then making $6 million in revenue and employed more than sixty people, disproving the fact that you cannot do business with open source. Larry McVoy was kicked out of Sun in the same year for advocating open source as a strategy.
- McVoy brought to Tiemann's notice a small company in North Carolina called Redhat - Tiemann advice to acquire a stake in that company was ignored and five years later, Redhat bought Cygnus.
- It took him two years to figure out how to do business with open source.
- Tiemann addressed the question of why OSS had succeeded and drew an analogy with the observation made by Alexis de Tocqueville in 1830s, that the sum of all individual undertakings in America exceeded the efforts of the government.
- Disruptive technology always comes from unexpected quarters.
- Cost cutting is not always the most interesting question - there is always more upside to inreasing revenue than to cutting costs. There are limits to cost cutting, while there are none to increasing revenues.
- The real value of architecture is when something comes along and it can be accomodated quickly. Explained the value of strategy with an example from a Wall St. firm - a strategy that was 10-20% different resulted in a ten-fold increase in growth.
- According to research done at CMU, the Apache project has ten to fifteen people who do eighty five percent of the work. But the total number of people who contribute is closer to 400 ( 388 ?). It's this extra non-core contributions that add lot of polish and quality to the software. Tiemann asked a rhetorical question about going to a venture capitalist to ask for money to recruit all 400 developers. (I thought this was a particularly effective illustration of the role of a large pool of contributors - more about this aspect in a future post, if time permits)
- Quoted Eric Von Hippel, faculty at MIT and the author of Democratizing Innovation, on the role of user participation in design. EVH in his book explains in his book how Jack Welch managed to make GE a leader in plastics by bringing in the concept of user design toolkits. Previously GE's business model made it possible to get into selected market segments, in fact the largest customers. Jack Welch said that GE doesn't need to be the exclusive designer of plastics - Why can't customers design their own plastics ? This idea of moving of moving the locus of design from the supplier to the customer resulted in 85% of the designs coming from the customers ultimately. MT then asked "What rational person would let go of this opportunity ?"
- Drew some analogies between disruptive open source technologies and themes from the book "Guns, Germs, and Steel" by Jared Steel. (The analogies about the conquest of the native american population by the Spanish seemed out of place to me)
- He recalled reading Stallman's code for the first time - first time he said, he saw good code. The architecture and implementation unfolded in one view.
- He remarked that even the newly elected board of the OSI did not have a clue about the reasons for the success of open source. That's the territory that needs to be protected and grown. He urged the audience and sophisticated users to think about it.
- Over 70% of all projects on sf.net are covered by the GPL or BSD style license.
- Quoted designer Bruce Mau about the fact that design is not very visible unless there is a failure and used Micrsoft's out-of-control spending on security to conclude that the shared shared model was a design failure. Microsoft's spending on security in Jul 2002 was $100 million and in Mar'2005 $2 billion. At this rate they should end up spending the rest of their bank balance on security. MT contrasted this with the improvements in open source software. The Fuzz report noted a 20% failure rate in Unix utilities in 1985. In 1990, by which time the GNU tools had appeared, the failure rate was down to 25% of that of Unix tools. In the same period the Unix utilties/tools had not improved substantially. Later the failure rate of GNU tools went down to 2% and by 1996 the GNU tools were a 100% clean.
- Software patents are the lead banner for massive stasis - it really means that innovation will stop for the next twenty years.
- Referred to Bruce Mau's work which rejects the notion of a client-designer relationship. The notion of an open source license may be obsolete, when it tries to break the above law.
- How to enage the plurality of potential contributions in OSS.
Update: Looks like Michael Tiemann gave a similar talk (pdf) at the Redhat summit.
Tuesday, July 05, 2005
Dave Winer on working groups
Dave Winer in a podcast explaining the evolution of RSS makes a interesting remark about working groups:
What happens when you put together a working group is that, you get people who like to be on working groups, not people who like to write software. And they have very different interests. They like to fly places, they like to have meetings, they like to discuss things, and discuss them again and again and again. They really like the discussions. They don't live to make the software. They want to make the discussions.
Subscribe to:
Posts (Atom)