Cycle Gap

John Gruber Has Some Career Advice For Developers

2011-11-15T22:06:00.010+05:30

(33:45-35:53 from John Gruber's keynote speech at The Çingleton Symposium, a conference which took place in Montréal on October 14th and 15th, 2011)

One simple way to look at it is that there are far more people who've never bought an iPhone and who've never bought an iPad, who will in the next five years than all of us who've already bought at least one to this point. And I don't see how anybody can deny that, unless something unbelievable, dramatic changes. That's certainly the way everything is going now. If you think this app store platform is big now, you really haven't seen anything yet. At an event last week, Tim Cook had a line - he said, 'This is an extraordinary time to be at Apple'. And He is definitely right. But I say to you, 'This is an extraordinary time to be an Apple developer' .This is the right time and the right place. This is a once in a career opportunity. This is like being a Rock and roll musician in the late sixties. This is like being a film maker in the seventies following Scorsese, Coppola, Steven Spielberg, George Lucas (when he was sane). If things go right, if things go the way I think they are going to go, these next five years, we are never going to work harder, we are never going to be under more pressure, we're never going to be more stressed, we are never going to feel like we have to work faster and we are never going to have to solve tougher problems. We're never going to have to move this fast. But the only thing any of us are going to regret is if we don't aim big enough. If you don't feel that you're now in a position to do the best work of your entire career, to look back and say, 'This was the time, I was there, I did this, I helped make this thing a reality', then you need to find a new position. This chance will never come again. And we are lucky, we're so unbelievably, incredibly lucky that it even came this once. Thank You.

Is A GitHub Fork A Real Fork ?

2011-04-17T17:14:00.007+05:30

Karl Fogel, who was most notably involved with the development of CVS & Subversion and is a director of the Open Source Initiative (OSI), makes an interesting observation about Github's usage of the term "fork" in an interview:

Actually, I think the notion of forking has not changed -- there has been some terminological shift, perhaps, but no conceptual shift.

When I look at the dynamics of how open source projects work, I don't see huge differences based on what forge the project is using. GitHub has a terrific product, but they also have terrific marketing, and they've promoted this idea of projects inviting users to "fork me on GitHub", meaning essentially "make a copy of me that you can work with".

But even though there is a limited technical sense in which a copy of a git-based project is in theory a "fork", in practice it is not a fork -- because the concept of a fork is fundamentally political, not technical.

To fork a project, in the old sense, meant to raise up a flag saying "We think this project has been going in the wrong direction, and we are going to take a copy of it and develop it in the right direction -- everyone who agrees, come over and join us!" And then the two projects might compete for developer attention, and for users, and perhaps for money, and maybe eventually one would win out. Or sometimes they'd merge back together. Either way, the process was a political one: it was about gaining adherents.

That dynamic still exists, and it still happens all the time. So if we start to use the word "fork" to mean something else, that's fine, but it doesn't change anything about reality, it just changes the words we use to describe reality.

GitHub started using "fork" to mean "create a workable copy". Now, it's true that the copy has a nice ability to diverge and remerge with the original on which it was based -- this is a feature of git and of all decentralized version control systems. And it's true that divergence and "remergence" is harder with centralized version control systems, like Subversion and CVS. But all these Git forks are not "forks" in the real sense. Most of the time, when a developer makes a git copy and does some work in it, she is hoping that her work will eventually be merged back into the master copy. When I say "master" copy, I don't mean "master" in some technical sense, I mean it exactly in the political sense: the master copy is the copy that has the most users following it.

So I think these features of Git and of GitHub are great, and I enjoy using them, but there is nothing revolutionary going on here. There may be a terminology shift, but the actual dynamics of open source projects are the same: most developers make a big effort to get their changes into the core distribution, because they do not want the effort of maintaining there changes independently. Even though Git somewhat reduces the overhead of maintaining an independent set of changes, it certainly does not reduce it so much that it is no longer a factor. Smart developers form communities and try to keep the codebase unified, because that's the best way to work. That is not going to change.

HN Comment: CEO Voice

2010-10-07T21:59:00.005+05:30

Some comments on Hacker News are so good, I've decided to start posting the interesting ones that I come across. No. 1, CEO Voice by seiji.

====== Normal conversation:

How was your weekend?

I dislocated a toe, slipped, then cut off a finger. It sucked spending the weekend in the hospital.

====== CEO conversation:

How was your weekend?

It was great! I had a new experience, got to meet some excellent doctors, and saw the inside of a state-of-the-art hospital too! I think we can do some cross promotional revshare with them. Just think about it -- every time you get sick, the hospital gives you a virtual item! We'll call it the Get Sick For Pixels campaign.

The Enterprise Software Business Summarised

2010-10-05T21:47:00.007+05:30

You don’t get it. The central relationship between Oracle and its customers is a business relationship, between an Oracle business expert and a customer business leader. The issues that come up in their conversations are business issues.

The concerns of developers are just not material at the level of that conversation; in fact, they’re apt to be dangerous distractions. ‘Developer mindshare’... what’s that, and why would Oracle care?
(via Tim Bray )

In other words, it's all about deals between managements.

Did Easy Money Derail Indian Software Product Companies ?

2009-10-31T18:19:00.014+05:30

Sridhar Vembu, the CEO of Zoho makes an interesting observation in a Hacker News discussion thread, about how easy money during the boom years prevented software product companies from taking shape in India.

"I have a different perspective on this, having built a product company (Zoho Corp) in India for the past 13+ years. Basically India's emerging IT industry coincided with the global credit bubble of the past 2 decades, which simply inflated the IT services industry to monster proportions, in the process sucking the oxygen of talent out of the system. So product start-ups that could have been never got off the ground.

This is the mirror image of the process in the US that funneled talent towards the financial complex, which meant the IT industry had to import talent or outsource the work. A good percentage of the talent came from India, and a lot of the outsourced work went to India. In that environment of easy money flowing into services, product companies found it hard to recruit talent (which the author of the post also alludes to).

This process basically derailed the emergence of product-based IT industry in India. Some of the Indian services giants of today (Wipro and HCL in particular) were product companies originally. They built some innovative products in India (an x86 based Unix well ahead of Linux, an IP concentrator/router for ISPs in early, to quote two examples) in the late 80s/early 90s. But once the 90's boom got underway, they found it far more lucrative to rent out the talent than to build products.

In Economics, it is impossible to argue "what might have been", but I believe the configuration of IT industry in India points to how the bubble massively distorted an industry."

I have been discussing the impact of easy money with friends for a while now and while it's a crucial factor, it's important to keep in mind that countries like Israel have chosen not to take the services route. The availability of easy money must have dovetailed with something else for software services (popularly known as body shopping) to take off in a big way in India. Dharmesh Shah, a successful serial startup guy makes some an great point in his post Why There Aren't More Software Startups In India:

At the risk of drawing stereotypes, I think Indians in general are a little impatient and like to see quicker “payback” periods on their investments; There are a few number of them (than in the U.S.) that are willing to spend the 2+ years it might take to build a product, see how the market responds and “tweak” the business as necessary to get it to a successful state;Product companies are also more “random” and difficult to control the outcome of;They involve a large number of “creative” factors that will largely influence whether the product succeeds or not. I’ve found Indians to be almost overly practical (in the short-term sense) and not passionate about some of the softer things (like user experience, marketing, branding and other things) which in today’s world are large contributors to future outcomes of software startups;They’d much rather work on the “harder” stuff that they can better control and predict;This is a bit of a “squishy” argument, but it’s a squishy issue;I guess the evidence of progress I’d like to see is a cool software product coming out of an Indian startup along the lines of an Adobe Photoshop or even a 37signals Basecamp.

An example of the impatience, inability to tweak the business, and overly practical attitude that Dharmesh talks about, is a Bangalore based company that's not too happy with it's clients for which it is currently developing and testing IPhone and Android applications (They inherited these customers after taking over a smaller company). The US based clients that outsource to them are mostly small to medium sized companies with relatively low billing rates. A lot of people would kill to get into IPhone and Android development now, but this Bangalore based company is very unhappy that they are not able to maintain their 60% plus boom era margins doing mobile application development. No attempt is being made to convert the expertise developed so far into a product making strategy. As a friend who works there observed, "They are only interested in projects that involve billing a hundred people or more".

Fred Brooks On COBOL's Success

2009-08-16T21:42:00.017+05:30

(~00:53:00) Question: In your slide on the conceptual integrity examples you made a passing reference to the ones on the left were not the ones with the fan clubs, but in many cases were the successful products. Given that, just how much value is there in the conceptual integrity if the successful products don't need it ?

Fred Brooks: Now, well, look, let's take COBOL - there were very strong forces to make COBOL market successful that had nothing to do with it's excellence. COBOL is a language, it was written, it was designed to be read, not written. It was designed so that bosses could see, could understand the code that people were writing. It is a committee design. It does not have conceptual integrity. But it had the department of defense mandating it, and so talk to me about market success when you have a DOD mandate when DOD is big customer. So there are many other factors other than the inherent excellence of the product to determine the whether it's a market success

(via podcast of Fred Brooks' keynote "Collaboration and Telecollaboration in Design" at OOPSLA 2007 )

Also at around 20:35:

Fred Brooks: How many people ever got delight from COBOL ?

Update (18'Aug 2009): The question seems to be motivated by a slide that has a table from the twentieth anniversary edtion of 'The Mythical Man Month'. Quote from the book (page 202):

A little retrospection shows that although many fine, useful software systems have been designed by committees and built by multipart projects, those software systems that have excited passionate fans are those that are the products of one or a few designing minds, great designers.

Fig. 16.1 Exciting Products (page 203)

Yes	No
Unix	Cobol
APL	PL/1
Pascal	Algol
Modula	MVS/370
Smalltalk	MS-DOS
Fortran

IT Startups And The Immature Indian Market

2009-08-11T17:35:00.010+05:30

There's lot of hand wringing over the lack of "real" startups in India. Most of the arguments totally miss the point that the market for technology products is barely incipient in India. Ashish Gupta nails it:

Ashish Gupta, Managing Director, Helion Advisors Pvt Ltd, feels it is difficult to sustain a company that sells technology because there isn’t a good enough market for technology in the country. Also, customers often prefer to buy solutions from established multinational companies rather than from an indigenous start-up.

‘Why should I buy IT from a start-up and add more risk to my business,’ is what many buyers ask, says Gupta. However, companies that sell services based on technology have far better chances of success in the Indian market, he adds.

(via The Hindu Business Line)

Why Google Owns Wave

2009-08-11T17:17:00.002+05:30

If you make it so complicated nobody else understands what you're doing, you own it.
- Dave Winer during a discussion of Google Wave during a 'Bad Hair Day' podcast

Compensating Users of Social Networking Sites

2009-08-11T16:49:00.005+05:30

It's amazing how little discussion there is about compensating users of social networking sites for user generated content, community activity, the free marketing via blog posts, tweets, etc - the primary response generally is to point out that the people running the site are not the only ones benefiting from it; Why would the user stick around if it was not a gainful or enjoyable experience in some way ?, and so on. I still remember feeling a bit peeved when reddit was taken over by Conde Nast - I was an early user and though my contribution was nothing earth-shaking , I couldn't help feeling that it was unfair. Dave Winer wonders why user's can't be compensated during a 'Bad Hair Day' podcast (~26:35) :

Take a guy like Scoble who pours his heart and soul into these products when when he really gets into it, why didn't a guy like that not get any upside. Why only employees of the company ? Why can't a user gets some stock ? I don't actually see why not.

Paul Graham On Two Kinds of Programmers and Painters

2009-08-06T20:33:00.007+05:30

In the world of painting there are some people who are just fabulously talented at drawing. They can sit down, these are like the kids who could draw in high school age fifteen they can sit down with a pencil ... sit in front of you ... wow, it looks just like me. But then when you say to these guys, "ok, use this amazing skill to just produce anything, just put it on the wall it's going to look great and then they lose". And within programming there is this distinction too. There are some people who are really really good at implementing code like if you give them a spec for a programming language and man, they will just implement it; The hardest stuff as long as you tell them precisely what to do, they will just do it. But you say, "ok, make up a product, make up some kind of new product that people want", and they are just utterly lost. This is actually a big mistake that companies make. There's a lot of companies who think that the programmers are basically implementers, that products are supposed to be designed by product managers. They are supposed to be designing what the products do. And they make mockups or something like that and they hand it to the programmers and the programmers translate their ideas into code. Like this one way process, no loopback - that loses! The best programmers are the ones that combine in one head both the ability to translate ideas into code and having the ideas. Just like the best artists have both the ability ... (have) a great hand. They can make their hand do what they want. But they also know what to tell it to do and actually between the two, I would take the Cézannes. Cézanne could not draw, he makes the same drawing mistakes that every one makes in introductory drawing classes. Occam's razor said he couldn't draw, not that he was trying to transcend three dimension ... But what he was good at was sort of the other half - deciding what to produce. He was terribly frustrated he was like this guy who had all kinds of ideas, but he couldn't articulate them with his hand. When you put the stuff on the wall in a room full of other paintings, it looks like there's a spotlight shining on his paintings and other ones have been sprayed with a light coating of mud. It's just amazing when you look at side by side paintings. So I will take the Cézannes actually and one interesting thing that has been happening is because programming languages have gotten so powerful you don't have to be that good an implementer to get something built.

- Paul Graham in an interview with Russ Roberts (~45:30-48:25)

The Pinnacle Of PR Success

2009-07-05T11:35:00.006+05:30

The pinnacle of success, as far as PR is concerned, is getting your marketing material into the NY Times, disguised as journalism.
- Owen Byrne

Owen Byrne is the guy who originally built Digg. Owen Byrne's comment is part of a Hacker News discussion thread about to an article N.Y.T piece "Spinning the Web: P.R. in Silicon Valley". Paul Graham's essay about how PR works, 'The Submarine' , is also insightful.

All honourable men

2009-06-27T15:27:00.004+05:30

Deccan Chronicle: You have, of course, resigned from Infosys. But have you had time to think about conflict of interest issues that might still arise?

Nandan Nilekani: The only time issues of conflict of interest will arise is during procurement. I will ensure that the UIDAI’s procurement is open and transparent. If need be, I will recluse myself from the decision-making process on procurements.
- From a Deccan Chronicle interview with Nandan Nilekani

The same day PTI reports that Infosys will bid for UID project:

Gopalakrishnan said Infosys would bid for projects under UIDAI like any other e-governance projects, but saw no conflict of interest though the authority would be headed by a former company top executive.
- From PTI report carried by rediff.com

Of course there is no conflict interest ! Why ? Because the MSM folks have told us repeatedly that Infosys executives are a bunch of demi-gods. Look at the amount of genuflection that's happening in Infy's neck of the woods. Yuck!

As the good old days of inflated head counts and billing rates comes to an end for Indian outsourcing companies, the big outsourcing companies will bid for, and push for huge local government projects. It's been happening for a while - the passport seva scheme of TCS is one such example.

Excessive 'Head Count' And 'Billing' Days Over For Indian Software Companies ?

2009-06-21T21:14:00.013+05:30

I was chatting with a friend at a mid-sized software company in Bangalore; he said that foreign clients were increasingly insistent that they would pay a fixed amount (typically $20/hour) for the projects he had been working on in the last year or so, regardless of the project member being a developer, manager, or tester. We were joking that software billing rates in another year's time would fall to McDonald's hourly wage. Anecdotal evidence from friends and ex-colleagues working on software projects for foreign clients in India seem to indicate severe reduction in hourly rates and head counts due to client pressure in the last one year.

Well everybody knew that it was a scam - the software boom in the nineties meant easy money for most outsourcing companies. I remember a cousin of mine mentioning the case of a Japanese client who was charged $8000 for changing the text on a dialog box. Not a single project ever ran without being overstaffed and the client being fleeced. There is enough schadenfreude to go around. While computers and software are supposed to make us more efficient and reduce unnecessary man power, Indian companies ironically used the software boom to increase the number of people employed. A person's worth at an India software service company was typically measured by "How many people report to you ?". Incidentally Sloka Telecom founder Sujai Karampuri also wrote "Why do we have so many jobs in Bangalore ?", which explains the dynamics of outsourcing quite well, though it's not comprehensive. The less said about the daylight robbery that went on in the name of ERP, the better.

A recent issue of Forbes (Indian Ed) covered the changes the management at Indian IT services bellwether Infosys is considering in the face of falling revenues. There seems to general agreement among the top executives that revenues based on project head count are getting squeezed and the company needs to find alternate ways of generating revenue.

...

Some indications of the emerging, new Infosys are already coming in. Ask Dhar. His elevation to the EC came just a year before the worst downturn in living history hit the world. It has been baptism by fire. And the decisions he has been taking have been unlike any at Infosys of the past.

Dhar found himself in a rather awkward spot with one of Infosys’ oldest customers — a telecom giant in midland Europe. The client was under serious pressure to reduce costs and the Infosys contract was in peril. Dhar knew he had to act fast. He was aware that a host of rivals had already offered to cut their price by 25-30 percent. If Dhar did not respond on time, Infosys would be edged out of other juicy contracts that were on the table.

He then did what would have been once unthinkable inside Infosys. Rather than reduce his per hour billing rate (which would have hit revenues and profits), Dhar decided to change the way the client was billed.

For a third of the value of the contract, he promised the client an upfront cost saving. Infosys would now charge the client on the number of technical problems it would solve instead of charging for the number of hours it worked in doing that. In sum, he was proposing an outcome-based model.

Now, the risks were considerable. To ensure that they still earned a profit from the account, Dhar had to bet on far fewer engineers than usual to use their native intelligence to pull off the job. But there was no guarantee that they would be able to accurately predict the rhythm of the job. In fact, in the first six months, the company might even lose some money working this way. But in the long run, Dhar believed it was the best thing for both the client and his company. “It’s always been at the back of our mind to move to this model, but we never had the incentive to do it. This downturn has given us the reason,” says Dhar.

...

Gopalakrishnan has put together a five point plan for getting Infosys ready for the next stage. This includes increasing the share of higher value services like consulting, delinking revenue growth from staff addition, pursuing large deals of above $500 million, improving efficiency and finding new locations for talent.

Update (23'June 2009): Hacker News reader edw519 makes a great observation in a comment on this post:

“It’s always been at the back of our mind to move to this model, but we never had the incentive to do it..."

You may have the incentive, but do you have the capability?

Fixed or value based contracts not only require better expertise, they require different culture. Even your good people have become do indoctrinated with the concept of fleecing the bill that they may have actually forgotten how to compete by simply getting the work done efficiently and effectively.

Good luck. Better yet, good luck to your customers.

Statisticians

2009-06-01T21:25:00.002+05:30

A couple of jokes about statisticians from Oxford mathematician Peter Donnelly's TED talk.

Someone, one of my senior colleagues told me when I was a youngster in this profession rather proudly that statisticians were people who liked figures, but didn't have the personality skills to become accountants.

And there is another in-joke among statisticians, and that is, how do you tell the introverted statistician from the extroverted statistician ? To which the answer is, the extroverted statistician is the one who looks at the other person's shoes.

Techie Spin

2009-05-29T21:37:00.001+05:30

It's always a shame when technologists, who have to answer precisely to the computer, use political spin when talking to users.
- Dave Winer

Rams' Law Of Enterprise Software Sales

2009-05-08T18:41:00.002+05:30

Never attribute to stupidity that which can be adequately explained by corruption.

Jim Coudal Interview (transcript)

2009-05-07T19:34:00.014+05:30

Jim Coudal (JC) in his interview with Ryan Carson (RC) offers valuable insights especially on the importance of being able to write well. Since it's a very short interview, I made an attempt to transcribe it, but couldn't make out all of it. Here it is anyway with the gaps marked with two red dashes '--'; words that I am not too sure about are indicated with a question mark (?). You are welcome to use this on your site/blog. If you can help me fill in the gaps that would be great.

RC: We are here with Jim Coudal from coudal.com and nice to have you here Sir

JC: Thank you, how are you (? )

RC: So why don't you tell me just a little bit about what's inspiring you right now

JC: It's sort of interesting. I have three children and the youngest of them is Spencer, he is seven and he has a kind of voracious appetite for re-living my childhood with star wars and everything else. So I don't know, maybe it's just the nature of the beast . But I am sort of inspired by childish things lately -- Spencer and my two daughters . Other than that I have been sort of wrapped up in vintage American agricultural design artifacts from the thirties and forties.

RC: How did you come across that ?

JC: Well we've this project called field notes, we sell these note books called field notes which is sort of based on that and one thing led to another and I started getting really into these(?) odd -- seed companies.

(1:05)

RC: Are you obsessed?

JC: I am constantly obsessed -- you know that about me (?). But this too shall pass, it will be something else. Yeah then you know we are doing the layer of tennis thing at Coudal, stuff that artists are coming up within fifteen minutes blasts is beyond me. I try to keep ...Yeah I know what's inspired. On Tuesday night we hosted Gary Hustwit's Objectified. Gary made Helvetica.

RC: Oh, wow !

JC: This new movie is about industrial design.

JC: He's on a kind of world rock and roll tour , London is coming up soon ...

RC: How wicked !

JC: In Portland -- and then I don't don't know and then maybe ... I don't know where he is playing. It is playing here in London and it's a great movie - Johnnie Ive from Apple, Dieter Rams from Braun really great movie really inspiring. These guys working inside of the industrial-manufacturing profit process and are total artists dedicated to details. Really well received.

RC: That was Objectified ?

JC: Objectified, that's my latest --

(2:06)

RC: How do you make time for creativity ? I mean you are a father, you have a successful business When does that happen ?

JC: For me I have sort of farmer's hours.I don't exactly know why. I get up early and once I get up I can't go back to bed. So generally I go to work in very early. I am there a couple of hours, before anybody else gets there. I find that couple of hours when I just spend that time working on something designy or creative writing, I am happy. So I might be setting a headline, I might be doing the layout for our website, I might be writing an essay - whatever it is, it's quiet, phone is not ringing, I am not answering email and then the rest of the day is pretty much acting and reacting. You know how it is, you know that I have things to do but there are things that I don't know I have to do (?) So I try to do that and then building light goes out.

RC: Cool. So if you can give a sort of a tip or an idea to a -- emerging (?) web designers either for their career, or probably for their career what would you say that is important ?

(3:11)

JC: I will give a tip for web designers in general is that (?), you know we are in an era where everybody is talking about sustainability and using only what we need to use. So leave a little white space . Don't use all the -- For a career tip I will just say this. When we are interviewing two designers for two creative positions and they are equally talented in every way and they will cost us equally as much and they are both friendly and happy and smart. But one of them can write and the other one can't, I will always hire her.

RC: -- (Got it ?)

RC: Good writing is a sign of an organized mind. As digital as we become writing is even more key than ever whether it's email, for writing copy for the web we find that we have done much better with even purely visual designers who have some writing skills. And I think it is under appreciated by the visual ...

RC: Agreed ...

JC: There's a day in the UK and in the USA as well where a creative ad agency was an art director and a copy writer together. Those days are gone

RC: Yes --

(4:26)

RC: -- you've to have the personality (?)

JC: I think you have to have the skills -- But I think you have to have both of those skills .
Maybe There's a third skill too and that's being able to write a little bit of code.

RC: Yeah.

JC: It's a sort of interesting time.

RC: So our designer Mike he seems to be an -- wrote all the lyrics - and I find that he is very
good at copy -- because he basically can, he used to write lyrics - there's something about
that ...

JC: Right

RC: Right

JC: I could see that, especially lyrics is generally trying to be as succinct as possible, succinct
and dramatic as possible . So it's probably good direction from -- copy -- succinct and
dramatic -- to get to the point -- keep people interested --

RC: Thanks for talking, see you soon.

JC: Sure.

How Chad Fowler Found Good Programmers in India

2009-02-26T22:11:00.004+05:30

In his essay GreatHackers, Paul Graham annoyed the industry with the assertion that Java programmers aren’t as smart as Python programmers. He made a lot of stupid Java programmers mad (did I say that?), causing a lot of them to write counterarguments on their websites. The violent reaction indicates that he touched a nerve. I was in the audience when his essay was first presented, in the form of a speech. For me, it sparked a flashback.

When I was in India weeding through hundreds of candidates for only tens of jobs, the interview team was exhausting itself and running out of time because of a poor interview-to-hire hit rate. Heads hurting and eyes red, we held a late-night meeting to discuss a strategic change in the way we would go through the candidates. We had to either optimize the people (or both). With what little was left of my voice after twelve straight hours of trying to drag answers out of dumbstruck programmers, I argued for adding Smalltalk to the list of keywords our headhunters were using to search their résumé database. But, nobody knows Smalltalk in India, cried the human resources director. That was my point. Nobody knew it, and programming in Smalltalk was a fundamentally different experience than programming in Java. The varying experience would give candidates a different level of expectations, and the dynamic nature of the Smalltalk environment would reshape the way a Java programmer would approach a problem. My hope was that these factors would encourage a level of technical maturity that I hadn’t been seeing from the candidates I’d met so far.

The addition of Smalltalk to the requirements list yielded a candidate pool that was tiny in contrast to our previous list. These people were diamonds in the rough. They really understood object-oriented programming. They were aware that Java isn’t the idealistic panacea it’s sometimes made out to be. Many of them loved to program! Where have you been for the past two weeks? we thought. Unfortunately, our ability to attract these developers for the salaries we were able to pay was limited. They were calling the shots, and most of them chose to stay where they were or to keep looking for a new job. Though we failed to recruit many of them, we learned a valuable recruiting lesson: we were more likely to extend offers to candidates with diverse
(and even unorthodox) experience than to those whose experiences were homogeneous. My explanation is that either the good people seek out diversity, because they love to learn new things, or being forced into alien experiences and environments created more mature, well-rounded software developers. I suspect it’s a little of both, but regardless of why it works, we learned that it works. I still use this technique when looking for developers.
- From Chapter 4 of Chad Fowler's book 'My Job Went to India: 52 Ways to Save Your Job'

Thanks to Pragmmatic Programmer for permission to use an excerpt from the book.

Proto.in 2009 - Striking the Right Notes

2009-02-13T22:28:00.002+05:30

I attended Proto.in, the two day event for startups in Bangalore in Jan'2009. Bangaloreinc has a couple of neat posts in their weekly roundup for 23-30'Jan.

Overall it was an useful and occasionally inspiring experience. For me the most heartening thing was how there are still people who care about ethics in the startup community, people who still think that the ends don't justify the means. At least they made all the right noises ... well, I shouldn't be really so cynical; watching some trends and hearing some of these folks does make one a bit more hopeful about the future of startups in India.This is important for me, having wasted my time in two startups that were out to game the system; I learnt at the conference that there is a term for this - 'lifestyle businesses' - startups that exist solely to enrich the founder(s). Another great thing was the remarkable absence of corporate drones at the event - there is something about about open source and startup events that makes them adopt a low profile despite their prominent sponsor banners.

A number of presenters struck the right note by honestly sharing their real experiences:

Allwyn of Pagalguy.com asked about the number of the huge number of Web 2.0 businesses that took money and produced nothing.
Ashsish gupta from Helion ventures gave a great talk that emphasised the need for founders to be intellectually honest
Atul Chitnis pointed out changed conditions, the economic downturn, the unworkable VC model, etc. Atul is the organiser of India's largest open source event foss.in and he seems to have begun his career by making DOS software.
Ashwin Mahesh with his tongue in cheek explanation of the IIM Bangalore incubation process easily made it one of the most entertaining presentations. It's so nice to see people from academia not take themselves very seriously. Oddly enough one of the organisers kept pointing out some government programs to help entrepreneurs - oxymoron and all that.
One of the most interesting facts that came out was the number of successful startup founders without an engineering or computer science degree. Suresh Sambandan said he had never been to college. Shalin Jain of ten miles has a B.Sc in Maths or Statistics. Looks like startups are relatively free of the tyranny of degree and pedigree that plague larger companies.
Bob Young gave a great talk - Ashish Gupta admitted it was a tough act to follow. Bob explained the history of Redhat and how he got started working from his wife's closet. I asked Bob what the big idea behind Lulu was. Bob explained that at Redhat he always had the nagging feeling that the contributors to the various pieces of open source software that Redhat was shipping were not being compensated properly. With Lulu he said they hoped to have a fairer compensation system for contributors.
The amount of soul searching that was evident in the panel discussions, the frank admission of mistakes, etc was remarkable. You don't get to hear this kind of honesty in vendor dominated software events or in internal meetings at larger companies.

The Indian Software Exports Scam

2009-02-01T19:40:00.008+05:30

The 12th Feb issue MoneyLIFE magazine has published a letter from an unnamed head of an Indian IT company. The letter does a great job of explaining why the whole software export business is a scam. About seven or eight years back, I was arguing with a friend that this whole 'Software Technology Park' (STPI) idea was not a good way to encourage software companies. A software company that's part of a government anointed STPI gets various concessions.

~~(Note: The letter is not online yet. I will update the post with a link later.)~~ Done.

The letter:

This is with regard to your cover story “The Truth about Satyam” (MoneyLIFE, 15 January 2009), and other reports on Satyam’s most recent adventures, which are not really new. Actually, the truth behind Satyam was well spelt out at the end of your report – what Satyam did was not way out of line – inferring that most other companies in India would be guilty of similar creative accounting and governance methods. But, with IT companies in India, there seems to be some justification trotted out by a variety of fellow-travellers, who bring in national prestige and credibility as an excuse for daylight robbery of the national exchequer and the public.

So, what’s at the root of this? How is it that companies that get the benefit of tax exemption worth thousands of crores of rupees now appear to be getting more money in thousands of crores from the government for, of all things, reviving the possibility of even more tax evasion? Never mind the diversion of profits that may also occur again. The answer is simple: a scheme called Software Technology Parks (STPI). This is a ‘scheme’ that has outlived its usefulness as far as real IT industries are concerned. But it seems to be around only to permit such scams. It is a fact that the STPI scheme, used by Satyam and misused by so many others, is at the root of the large number of IT companies in India, declaring huge tax-free profits and then going under or simply vanishing. By modest estimates, over 96% of the companies registered under STPI came in, took their tax exemptions on imports, took some more tax exemptions on export profits, and then disappeared. A simple analysis of the top-20 lists with NASSCOM over the last decade will provide evidence for this. A slightly more complex examination of the way the STPI scheme is actually performing will give an even better indication that things are going terribly awry.

From being a facilitator and an incubator for IT companies, the STPI scheme has become yet another den of inspectors and officials who, in league with other government bodies, are there simply to permit all sorts of duty-free imports and tax-free exports. Since real IT exports don’t really need this exemption any more, it is only the scamsters who increasingly benefit. Simply put, buying computer hardware is now cheaper if done directly, without using import duty exemptions. The real game here, as always, is in over-invoicing; and non-existent imports followed by equally non-existent exports.

The second important parameter which needs to be brought out is the way headcount is multiplied, for assorted benefits especially towards showing and booking higher expenses on payroll, allied expenses and more – the more employees you show, the higher these can be. And, in the corresponding billing as well as dummy profits, more fictitious employees mean higher billing and, therefore, higher paper profits. A simple double-check would involve finding out how many employees such IT companies have registered with the EPFO (Employees’ Provident Fund Organisation); but that’s often not easy to do because the EPFO’s own records are in tatters too. Just one number should suffice. Satyam claims to have 53,000 employees; but, at their second largest centre in Pune, they have just about 4,500 employees. Where are the rest? It simply doesn’t add up.

Satyam is not a one-off or stand-alone scam. There are many like them and one would have presumed that those in charge at NASSCOM would have taken action against such members of their association well in advance. The fact that they haven’t, makes one suspect that all is not well with NASSCOM too. And, therefore, it is all the more amazing that elements from NASSCOM have been put in charge of Satyam’s revival. To fix such problems, one has to go deep into the roots and pull out the rot. That will happen only when tax exemption of expenses as well as revenues for the IT sector is done away with. Otherwise, like the fertiliser sector, the tax-exempt IT companies will simply be like addicted sucklings not being weaned off the mammary of the State.

The writer heads an IT company and does not want his name to be disclosed. – Letter by email

One Reason Why Software Is Expensive

2008-11-30T16:21:00.003+05:30

Joel Spolsky recently spoke at Y Combinator about selling software to corporate customers. He said that in most companies software costing up to about $1000 could be bought by individual managers without any additional approvals. Above that threshold, software purchases generally had to be approved by a committee. But babysitting this process was so expensive for software vendors that it didn't make sense to charge less than $50,000. Which means if you're making something you might otherwise have charged $5000 for, you have to sell it for $50,000 instead.

The purpose of the committee is presumably to ensure that the company doesn't waste money. And yet the result is that the company pays 10 times as much.

- Paul Graham in 'The Other Half of "Artists Ship"'

How Fortran Was Developed

2008-11-24T20:37:00.006+05:30

While going through a presentation (pdf) by John Anderson of the University of Edinburgh at LISA'08, I came across a fascinating quote:

As far as we were aware, we simply made up the language as we went along.We did not regard language design as a difficult problem, merely a simple prelude to the real problem: designing a compiler which could produce efficient programs.
- John Backus, Developer of Fortran and inventor of BNF

Googling a bit led to Backus' paper (pdf) from which the quote had been taken.

From The Horse's Mouth: The Free Desktop Has Been In 'Catchup' Mode

2008-11-22T15:28:00.002+05:30

The free desktop has been in “catchup” mode: catching up to first Windows and now nipping at the heels of the Mac. Our path has been obvious to date. In some areas, our technology and applications lead; in others we still lag. From here on, progress becomes much less clear, though I’ll bet on the moving herd and natural selection of free software over directed closed commercial development any day.

How now to move from such a reactive strategy to true leadership in all areas? How do you set strategy, when our very culture is that of serendipity, discovery, sharing of ideas, and creation? where a single vision cannot rule?

- From Jim Gettys' post 'Time To Lead ...'

In case you are a newbie or one of those irrational defenders of crappy open source desktops, please check out the Wikipedia entry for Jim Gettys. No wait, I will save you that click with this one line from that Wiki entry:

He is one of the original developers of the X Window System at MIT and worked on it again with X.Org, ...

Andy Bechtolsheim On a Common Start-Up Mistake

2008-10-25T15:01:00.003+05:30

Lean staffing also helps Arista keep its costs down. The Menlo Park, Calif., company has fewer than 50 employees and started shipping systems a few months ago even though it had no formal chief executive.

“One mistake a lot of start-ups make with the encouragement of venture capitalists is to hire the whole management team upfront,” said Mr. Bechtolsheim. “You have a lot of people twiddling their thumbs and spending money.”

- via NYT article Sun Loses Co-Founder to Start-Up

A Marketing Guy Nails the Problem with Software Companies

2008-10-25T14:41:00.007+05:30

Watching Steve Johnson's short six and half minute presentation titled 'Software:Business or Hobby' (video) at the Business of Software Conference (2007), I was struck by something he said (around 04:13):

One of the challenges, I think that we all face is so many people in the organization are making decisions about that they are not qualified to make decisions about. Nothing seems hard to the people, who don't know what they are talking about.

- Steve Johnson of pragmatic marketing

You can watch the whole presentation here.

Mercurial vs Git: The Biggest Non-Technical Difference

2008-10-18T18:24:00.001+05:30

The biggest non-technical difference between git and mercurial is the rabid culture surrounding git. mercurial users fairly happily and quietly use their tool, while I've had to send two separate door-to-door git missionaries away today alone.
- Dustin Sallings

Software Development - Fire Fighters vs Real Heroes

2008-10-12T17:33:00.004+05:30

In a Dilbert cartoon, the pointy-haired boss, apparently frustrated by the company's sub-par products, announces that he'll reward each bug fix with a $10 bill. Wally says: "Hooray! I'm gonna code me a minivan!"

Unfortunately the heroes, those who seem to save the organization in a great flurry of activity, are often reacting dramatically to the problems they created. Like Wally, they're rewarded for the successes while no one notices that furious activity is no substitute for doing things carefully.

Solving problems is a high-visibility process; preventing them is much better, but earns few rewards. This is illustrated by an old parable:

In ancient China, there was a family of healers, one of whom was known throughout the land and employed as a physician to a great lord. The physician was asked which of his family was the most skillful healer. He replied, "I tend to the sick and dying with drastic and dramatic treatments, and on occasion someone is cured and my name gets out among the lords.

"My elder brother cures sickness when it just begins to take root, and his skills are known among the local peasants and neighbors.

"My eldest brother is able to sense the spirit of sickness and eradicate it before it takes form. His name is unknown outside our home."

Unfortunately, sometimes the very best developers get the least acknowledgement, even from their own teams.
- Jack Ganssle, Embedded development expert

The Programming Elite, Programmers Who Read

2008-08-31T17:37:00.002+05:30

About half an hour into the StackOverflow podcast, I started wondering whether I should stop listening to all the navel gazing about the soon to be launched Stack Overflow. Then it got a little more interesting when the discussion veered to the kind of readers/listeners that the site might get after the launch. Joel Spolsky makes an interesting comment (around 29:17):

"The audience of people that read Coding Horror and the audience of people that read Joel on Software are already fairly elite in the programmers, because they are the kind of people who read things in order to better themselves as programmers. And that's already, you know, 5-10% of practising programmers. It's not the vast masses of Java monkeys who were formerly VB monkeys who were formerly COBOL monkeys who are just doing, you know, large swathes of extremely boring stuff internally somewhere. Ahh, Who have I not offended ?"

Joel also adds a little later, "Don't bother writing in, I will just commit suicide."

Seriously though, Joel's comment struck a chord with me. The programmers who read online, especially technical stuff unrelated to their work are a minority. The ones who read books are an even smaller group. Tom DeMarco and Timothy Lister's classic "Peopleware" has the following to say about reading habits of programmers

"The statistics about reading are particularly discouraging: The average software developer, for example, doesn't own a single book on the subject of his or her work, and hasn't ever read one. That fact is horrifying for anyone concerned about the quality of work is the field; for folks like us who write books, it's positively tragic."
- From Peopleware, Productive Projects and Teams by Tom De Marco and Timothy Lister (2nd Ed, page 12)

Paul Graham On Why Some Popular Web Sites for Nerds Went Downhill

2008-08-20T22:15:00.006+05:30

I don't think it's as hard to keep a site from sliding as one might think from the examples of previous sites where things went downhill as they got more popular. Slashdot, Digg, and Reddit were all companies. They wanted to grow. Whereas News.YC is a side project. We don't care about growth. It's much easier to do things to keep up the quality when you're willing to sacrifice growth.
-Paul Graham in a comment on Hacker News

Steve Jobs On Recruiting

2008-08-15T10:28:00.002+05:30

"Recruiting is hard. It's just finding the needles in the haystack. We do it ourselves and we spend a lot of time at it. I've participated in the hiring of maybe 5,000-plus people in my life. So I take it very seriously. You can't know enough in a one-hour interview. So, in the end, it's ultimately based on your gut. How do I feel about this person? What are they like when they're challenged? Why are they here? I ask everybody that: 'Why are you here?' The answers themselves are not what you're looking for. It's the meta-data."

- From an interview with Fortune

Professional Stereotypes and High Performers

2008-08-09T17:18:00.003+05:30

In response to Paul Graham's essay A Fund Raising Survival Guide, Gordon Mohr has posted a fascinating insight in the comments:

The very last sentence of the last footnote caught my eye:
Oddly enough, the best VCs tend to be the least VC-like.
I suspect that rather than being odd this is nearly tautological for any profession -- "the best X tend to be the least X-like". Professional stereotypes are set by the multitudes in the middle, not the highest-performing outliers.
Further, atypical behavior can be both a cause and effect of excellence. Being 'different' helps them be 'better', but also by being 'better' they gain freedom and confidence to deviate from norms.
(Of course, "the worst X tend to be not very X-like" is also true. But they're more likely to at least try to emulate the average X.)

- (via Hacker News)

Joshua Schachter on the Yahoo del.icio.us Rewrite

2008-08-09T16:13:00.003+05:30

The old, perl codebase was a pain -- I should know, I wrote it.

...

of course, this was just integration with an off the shelf search solution that was available internally. but engineering was focused on the rewrite so nobody had time to replace the old search engine (xapian, which did an admirable job, but not at that scale)

...

The writer is accidentally correct - we were told that it had to be in PHP to get ops support. Curiously the PHP part didn't take that much time - the majority of the "business logic" is in C++ which took forever and ever to write. I think the open question now is whether the remaining team will be able to innovate or be stuck in complicated codebase hell.

...

Right. The PHP part was not the time sink. On the other hand, the general rewrite took forever.

...

They had the technology, but the relevant staff had higher priorities.

(via reddit)

While it seems to be par for the course for big companies to screw interesting start-ups after a take over, these comments are pretty revealing all the same. Rewrite software to make it easier for ops ? As more and more software moves to the data center, ops is becoming one of the potential pain points and political power centers. I have been ranting about this to a couple of friends and don't seem to be getting my point across. I think one way out of this mess is a new breed of programmer administrators and automation tools like Puppet.

IBM Executive To Linux Desktop Developers: 'Stop Copying Windows'

2008-08-08T23:04:00.006+05:30

IBM, whose decision to back Linux years ago was a driving force in its adoption by business, called on developers of the open-source operating system to make it more "green" and to stop copying Windows, if they want to see Linux on the desktop.

Bob Sutor, VP of open source and standards at IBM, told attendees of the LinuxWorld Conference in San Francisco, that what the open source community needs to make Linux popular as a desktop OS used by consumers and businesses are "some really good graphic designers."

"Stop copying 2001 Windows. That's not where the usability action is," Sutor said during his afternoon keynote.

- Information Week

Wow ! I suspect most reactions are going to be predictable. The real problem though, might be that "some really good graphic designers" are not the only factor that could result in a great Linux desktop. Linux development is now heavily influenced by corporations that are not known for any great user facing design efforts. Great UI work does not seem to lend itself to the kind of distributed development style that has made many open source projects successful. Successful open source projects which are usable by the ordinary user are few and far between. Good UI requires a different mindset as John Gruber explained once in an excellent post:
UI development is the hard part. And it’s not the last step, it’s the first step. In my estimation, the difference between:

software that performs function X; and
software that performs function X, with an intuitive well-designed user interface
isn’t just a little bit of extra work. It’s not even twice the work. It’s an entire order of magnitude more work. Developing software with a good UI requires both aptitude and a lot of hard work.

...
It’s not something every programmer can learn. Most programmers don’t have any aptitude for UI design whatsoever. It’s an art, and like any art, it requires innate ability. You can learn to be a better writer. You can learn to be a better illustrator. But most people can’t write and can’t draw, and no amount of practice or education is going to make them good at it. Improved, yes; good, no.

Conversely, some people who are good UI designers aren’t programmers. But the rock stars are the guys who can do both, and they are few and far between.

If there’s a glib, nutshell synopsis for why Linux desktop software tends to suck, it’s this: Raymond and his ilk have no respect for anyone but themselves.

They have no respect for the fact that UI design is a special talent.

They have no respect for the fact that good UI design requires a tremendous amount of time and effort.

And, most importantly, they have no respect at all for real users. The idea that GUI software needs to be designed for “dumb users” — which is Raymond’s own term, and an indication of what he really means when he refers to dear old A.T. — is completely wrong.

Great software developers don’t design for morons. They design for smart, perceptive people — people just like themselves. They have profound respect for their users.
...
Movies are collaborative art, but require strong direction. So it is with end user software.

Gruber's post is so good, that I run the risk of reproducing it in its entirety.

Funny Perl Quote

2008-08-04T22:50:00.005+05:30

My only argument against perl would be it looks like a chicken ran across your keyboard.

- Michael Whalen's comment on a blog post titled In defense of Perl

Linux Kernel Development Stats from Greg Kroah Hartman

2008-07-12T22:58:00.008+05:30

Linux kernel hacker Greg Kroah Hartman's June 5, 2008 talk at Google titled "The Linux Kernel" was chock-full of details about kernel development. I noted down some of the the things he said. Please note that the talk was delivered on June 5th, 2008 and all stats mentioned by GKH are relative to that date.

Code changes per day in 2007-2008 so far
- 4,300 lines added
- 1,800 lines removed
- 1,500 lines modified
These changes don't include moving files around.It works out to about 3.69 changes per hour, 24x7 . It's not just the drivers that are changing at this rate, it's the the entire kernel.
The kernel itself is about 9.2 million lines, and has been increasing in size by about 10% every year since 2.6.0 (when GKH started tracking it). The drivers make up about about 55% of the code, while architecture specific code is second in terms of LOC. The core of the kernel is about 5%.
Supports more processors and devices than any other OS in history.
One of the consequences of the scorching pace of the kernel development is that, the kernel is a better place for patches , than for it to be maintained separately. GKH pointed out that it is difficult to keep pace with Linux kernel development and gave the example of Xen; The Xen folks apparently never played nice with the kernel developers and are now trying to get their patches into the kernel. From his tone it sounded as thought it was a tough climb uphill for the Xen guys, and the KVM is already in the kernel.
The Kernel development hierarchy (it has one !) has developers feeding patches to the maintainers, who in turn feed them to subsystem maintainers (pci, security, usb, etc).Sub-system maintainers maintain their own trees. Steve Rothwell (IBM, Austalia) pulls changes into the "Next" tree every night and does daily builds, while Andrew Morton pulls changes into his tree once a week or so. When Linus says the merge tree window is open, all the sub-system maintainers hit Linus. There is also the stable release tree, which is not maintained indefinitely.
He likened the patch submission process to a lossy network routing algorithm - it can handle people dropping out . There is no other way to develop at such high speed.If you own a file or subsystem, you have to accept the fact that other people are going to be changing it. Maintainers can always revert changes if they don't like it.
There is no good way to test the kernel except to run it. The hundreds of permutations of devices and interactions makes it impossible to test it comprehensively. The only way out is for the developers to test the rc releases.
There are no stable and unstable releases anymore. For the last four years,they have been replaced by releases every 2 and 3/4 months.
There have been 2399 independent contributors to the Linux kernel in the last year. 50% of the contributors submitted only one patch; half of the half contributed two patches. the top of the curve is getting flatter. Top 30% do only 30% of the work. The number of individual contributors is going up.
Top developers by quantity (for the last one and half years)
1. Adrian Bunk 754
2. Al Viro 698
3. Thomas Gleixner 656
4. David S.Miller 655
5. Bart Zolnierkiewicz 637
6. Paul Mundt 610
7. Ralf Baechle 604
8. Ingo Molnar 596
9. Patrick McHardy 554
10. Tejun Heo 530
Top contributors by sign-offs (shows who the gatekeepers are). Note that Linus doesn't sign off on patches from subsystem maintainers.
1. Andrew morton 9086
2. Linus Torvalds 8960
3. David S.Miller 4926
4. Jeff Garzik 2960
5. Ingo Molnar 2489
6. Greg Kroah-Hartman 2098
7. Thomas Gleixner 1098
8. Mauro Carvalho Chehab 1822
9. Paul Mackerras 1675
10. John Linville 1461
Who's funding Linux kernel development ?
1. Amateurs 18.5%
2. Red Hat 11.6%
3. IBM 7.5%
4. Novell 6.6%
5. Unknown individuals 5.5%
6. Intel 4.1%
7. Oracle 2.2%
8. Consultants 2.2%
9. Academia 1.5%
10. Renesas Technology 1.5%
Google is at number 13 with 1.4% contribution. Without Andrew Morton's contributions Google's would be at the fortieth spot.
Canonical had about 6 changes in the past 5 years; they are in the 300th
position. GKH was very emphatic that 'Canonical does not give back to the community'.
75% of Linux kernel work is paid for.
Linus jokes that the kernel is not intelligent design, it is evolution. GKH also added that "We react to stimuli that's happening in the world" and "We don't over plan things".
As GKH put it, 'We broke all software development rules and we are continuing to break it'.

The Importance of Apple

2008-06-01T09:34:00.004+05:30

Recently at work I couldn't help chuckling and shaking my head as I went through a list of requirements for a well known MNC - buried in the middle of was 'Oracle support' - there, that's an enterprise customer, I thought. It's one of those 'enterprise' requirements that clients who work in multi-storeyed glass and concrete buildings with large cube farms feel concerned about regardless of whether it makes any real difference to their work. This was not the first time that I had heard users asking for Oracle support. It's all about making the right noises, which is what enterprise software is all about. I was more annoyed with the business development folks for not having nipped this in the bud than with the customer. A couple of days later I was reading an interview with the author of Cocoa Programming for Mac OS X when I came across this heart warming section:

Scott: The Mac appears to be making real inroads in the mainstream consumer computing market, and certainly the iPhone is doing the same. Do you expect to see this carry over to the corporate world?

Aaron: Apple seems to doing its best to keep Macs out of the corporate world. Most dialogues between Apple and a corporation go something like this:

Corporation: Hey, does Cocoa include something to help us write apps that talk to our Oracle database?

Apple: Um, nope. But isn't GarageBand cool?

Corporation: Hmm.

Indian IT Companies and Internet Access at Work

2008-02-27T10:28:00.004+05:30

Shyam points out the hypocrisy of a TCS executive advocating net access at work while denying the same to his own employees. A couple of years back one of my ex-colleagues along with one of the sales guys went to TCS to make a sales pitch. Using our product meant all your data was hosted off-site; the data could be accessed using a browser based interface, as well as command line tools and GUIs. They spent a couple of hours explaining the details of this product, and answering all kinds of questions about edge cases and close to impossible usage scenarios. They were feeling a little exasperated about the oddness of the questions, especially, since they were informed that this was an audience of senior managers. Finally they learnt that it would be difficult for the TCS staff to use the product, since they did not have Internet access. My ex-colleagues were more than a little peeved about their time being wasted.

IT work in recent years has become very dependent on access to search engines, online forums, mailing lists and IRC groups. Why try to fix something from first principles, when you can paste the error message into a search engine and get an easy answer ? Makes sense in most cases, especially crappy software, which means most software. It makes little sense to attempt to understand the workings of twisted minds that develop such software. Add to this the amount of open source software in use at these days. The very culture of open source means you will never really be able to use it meaningfully without internet access.

Management concerns about internet access cannot be wished away either. It seems to revolve around a couple of key concerns.

Downloading and installation of various kinds of malware by employees and subsequent IT resources and time spent on cleaning up the network.
Browsing sites unconnected to work and chat. I have to admit that, I find the amount of time that some recent graduates spend on social networking sites and chat borders on the pathological. And it's not always for lack of work to do. It's a huge time sink.
Legal obligations of employers - what if the employees are surfing porn sites ? That could be construed as sexual harassment by their female colleagues. Isn't the employer liable in this case ?
Data security has become a huge concern with management - nobody seems to have a good solution to this problem. There seem to be a few back office type operations that apart from blocking net access disable access to USB drives and such. There have been quite a few scams that seem to have made managements paranoid.

Employer trust is not something that is always reciprocated by employees. A liberal employer is often misunderstood to be a weak personality, who can be taken advantage of. Misuse of office facilities is a reality that cannot be wished away.

There are no hard answers to a complex techno-social problem like this. An evolutionary approach is needed that takes into account the concerns of both employers and employees. Employers need to realise that, increasingly, internet access is not a priviliege that they 'allow' employees to enjoy; In many cases such as software development it helps to work effectively. Allowing employees to finish domestic chores such as paying bills, check their bank accounts, or keep in touch with their friends and relatives (in moderation) contribute to productivity at work.

I once heard from a consultant in Bangalore, that his client not only had him setup a proxy server for net access, but also had him setup an intranet site where everyone's browsing records were publicly available. This seems like an interesting approach that could work in some cases. In any case I don't have a problem with employers monitoring my internet activities. Employers should make their policy on net connectivity very clear; what's acceptable and what's not should be explained; new employees should have it clearly explained to them that net access is not a 'given' and that their activity is being monitored. Most companies do not have a policy on this matter and their only solution is a blanket ban on any kind of access, with a patronising common 'net PC' thrown in.

OLPC - Trying to make Water flow Downhill

2008-02-27T10:10:00.002+05:30

I don't understand why we need a program to spend millions of dollars to make laptops cheaper, when laptops get cheaper everywhere. I mean, that would be like setting up a world bank program to make water flow downhill.
- Dr. Joel Selanikio co-founder of DataDyne in an interview with Jon Udell

That must be one of the funniest and most insightful comments on the OLPC project. I suspect it's actually one of the most nagging questions that bothers folks who are not evil.

Virtualization, The Jamie Zawinski Remix

2008-02-26T22:08:00.006+05:30

Some people, when confronted with a problem, think "I know, I'll use virtualization." Now they have two problems.

(Jamie Zawinski - Wikiquote)

Automated Testing and Programming Skills

2007-12-27T21:28:00.000+05:30

Joel Spolsky's lecture at Yale university makes some interesting points about the dangers of automated testing. Joel is good at latching on to the unintended consequences of latest trends, especially the kind that nerds with Asperger's typically overlook. He speculates that 'automatic testing religion' was what did Vista in. There is an interesting bit about the fate of testers without programming skills at Microsoft:

Microsoft has had a long term policy of eliminating all software testers who don’t know how to write code, replacing them with what they call SDETs, Software Development Engineers in Test, programmers who write automated testing scripts.

I can sympathise with this, having interacted with testers without programming skills myself. If Microsoft's policy were to be adopted in India, there would be quite a few unemployed testers. Joel's article is a helpful reminder of the dangers of overdoing automated testing. But I think the situation in India represents the other extreme - we have lots of testers without any real skills just trying to fill up bug trackers and warming chairs in head-count based projects. Automation is the last thing that concerns either testers or software companies in India. After the dot-com bust, testers have become an overvalued commodity in Indian software companies with clueless managements.

John Gilmore's test of a good programmer

2007-12-12T10:57:00.000+05:30

John Gilmore suggested to me a test of a good programmer: one who has written a piece of software that at least 1000 people have downloaded from the Internet.

- Arun Mehta

OLPC Laptop Application Virtualization

2007-11-22T18:21:00.000+05:30

I was listening to a fascinating presentation (video)by Ivan Krstić, the director of security architecture of the One Laptop Per Child (OLPC) project. What struck me most was the extent to which virtualization is being used. Unless you have been living under a rock, you know that virtualization is a 'hot' area now. But this must be one of the few attempts to use it to provide desktop security (Incidentally Anti-virus tools routinely use virtual machine containers) The Bitfrost security model involves running each application in its own virtual machine container. As the Wikipedia entry explains,

Every program, when first installed, requests certain bundles of rights, for instance "accessing the camera", or "accessing the Internet". The system keeps track of these rights, and the program is later executed in an environment which makes only the requested resources available. This is implemented by a fully-fledged, container-based virtual machine.

By default, the system denies certain combinations of rights; for instance, a program would not be granted both the right to access the camera and to access the internet. Anybody can write and distribute programs that request allowable right combinations. Programs that require normally unapproved right combinations need a cryptographic signature by some authority. The laptop's user can use the built-in security panel to grant additional rights to any application.

Running each application inside a virtual machine is a scary idea, especially given the hardware configuration of the OLPC laptop. Krstic explained that the Linux kernel VServer patch had been extended to do OLPC specific tasks. Perhaps, the most fascinating piece of information that I learnt from the whole talk was the low overhead associated with such Linux VServer virtual machines. Quote from Krstic's presentation:

"The interesting thing about this, by the way is you know people are terrified of how are you going to do virtualization with 466MHZ CPU - turns out with Linux Vserver that the overhead you pay is 32 K per task struct, but there is 0% measurable CPU overhead with up to 65,000 Virtual Machines running ."

I have mixed feelings about this approach. Having done virtualization related work for the last two years, I find this approach fascinating;but, it could also start a disturbing trend of people opting for this approach instead of trying to solve security problems at a more fundamental level. But I got to admit that the clever Copy-on-Write approach of the VServer just blew me away. As Krstic said after explaining the low overhead associated with the virtual machines,

"I will let that sink in for about two seconds"

It's not as bad as the Sendmail configuration file

2007-11-09T08:30:00.000+05:30

But the Apache web server configuration file is pretty close. I once heard one of the founders of the Apache project confess that, the only configuration file worser than it was the one used by Sendmail; I mean, that's not saying much, is it ? I wasted the whole afternoon yesterday wrestling with the VirtualHost directive. Finally, I threw in the towel and got it working in five minutes with twisted web. It's httpd.conf that's twisted. All I wanted was a couple of web servers on my laptop to test my program. The worst part - it makes you feel so stupid for no fault of your own.

The Glorious History of HR in Software Companies

2007-09-15T21:31:00.000+05:30

Mitch Kapor's wife Freada was in charge of HR at Lotus in the early years. (As he is at pains to point out, they did not become romantically involved till afterward.) At one point they worried Lotus was losing its startup edge and turning into a big company. So as an experiment she sent their recruiters the resumes of the first 40 employees, with identifying details changed. These were the people who had made Lotus into the star it was. Not one got an interview.
- Paul Graham in a footnote to his essay News from the front

Extreme Pair Programming - Guy Steele and Richard Stallman

2007-09-11T20:49:00.000+05:30

"We sat down one morning," recalls Steele. "I was at the keyboard, and he was at my elbow," says Steele. "He was perfectly willing to let me type, but he was also telling me what to type.
The programming session lasted 10 hours. Throughout that entire time, Steele says, neither he nor Stallman took a break or made any small talk. By the end of the session, they had managed to hack the pretty print source code to just under 100 lines. "My fingers were on the keyboard the whole time," Steele recalls, "but it felt like both of our ideas were flowing onto the screen. He told me what to type, and I typed it."
The length of the session revealed itself when Steele finally left the AI Lab. Standing outside the building at 545 Tech Square, he was surprised to find himself surrounded by nighttime darkness. As a programmer, Steele was used to marathon coding sessions. Still, something about this session was different. Working with Stallman had forced Steele to block out all external stimuli and focus his entire mental energies on the task at hand. Looking back, Steele says he found the Stallman mind-meld both exhilarating and scary at the same time. "My first thought afterward was: it was a great experience, very intense, and that I never wanted to do it again in my life."
- Guy Steele on a hacking session with RMS in the 1970s.

Beautiful Code - The Paris Hilton Connection

2007-08-18T22:19:00.000+05:30

The odds of finding truly beautiful code in most production systems seem to be on par with the odds of finding a well-read copy of IEEE Transactions on Software Engineering in Paris Hilton’s apartment. Furthermore, enterprise software, which represents the bulk of code in existence, deals mostly with forms, reports, etc., which – one might argue – seldom require much thinking or cleverness.

- Albert Savoia in a post on O'Reilly's Beautiful Code blog

It's not just enterprise software - you find bad code in lots of startups, where there is an exclusive focus is on getting things done rather than doing it correctly. In fact, even lots of open source code bases are full spaghetti code. A great hacker even recommends FreeBSD code over Linux code.

Sadly, a lot of Linux and Gnome open source is poorly written.

- Trevor Blackwell in response to 'How can I become a better programmer ?'

Startups are not Democracies

2007-06-09T13:53:00.000+05:30

If you are not one of the founders of a startup, you really need to aware of this. I wish someone had reminded me about this :-( .

Startups are not a democracy. Want a democracy? Go run for class president, Bueller.
- From Mark Fletcher's 15 Startup commandments

The Dirty Secret of Blogging

2007-06-09T13:48:00.000+05:30

That's one of the dirty little secrets of blogging: the discussion in the comments are often better than the original entry. - Jeff Atwood

Simplicity, a feature

2007-06-09T13:37:00.000+05:30

Recently, at work, I had to use the XML-RPC.Net library and couldn't help smiling at this tongue in cheek answer in the FAQ section.

Why use XML-RPC instead of SOAP?

If your clients and servers are all running in the .NET environment there is no point in using XML-RPC: .NET provides excellent support for SOAP and XML-RPC doesn't have any features not provided by SOAP (other than simplicity).

- XML-RPC.NET FAQ

Why Developing Software for Indian Companies is Not Easy

2007-03-27T21:54:00.000+05:30

I was reading an article Indian IT biggies losing out to MNCs, when I came across a very interesting and insightful comment by Chanakya. Yet another instance where a comment on an article is far more interesting the article itself. A couple of years back, I quit my job since I couldn't not handle the irrational pressures involved in developing for certain Indian customers - I quit even without having another job in hand. I know of a very idealistically motivated software company that had created a great ERP product for the SMB market - the last I heard, they have stopped trying to sell to Indian customers and were focusing their sales efforts mainly on the US and Europe. It's not for want of idealism, or lack of technical competence, or domain specific competence that you find very little serious software development work targeted at the Indian market. Most of the the fine folks who have tried are suffering from Indian IT market burn-out, for the reasons for which are so eloquently explained by, Chanakya:

It is very true that the Indian IT companies are failing to corner big deals in the domestic market. Both the IT companies and the domestic clients are to be blamed for that.

Let us first start with the Indian clients. Their IT awareness is quite poor. Their business processes are also not very much IT-dependent. In fact most of them view computerization as an additional burden - a sort of necessary nuisance. The business processes of those companies are so non-standard that it becomes rather difficult for them to get the real benefits of IT. Today's well evolved softwares require clearly defined and standard business processes and, most important, discipline as pre-requisites. For example, to get the optimum performance out of a Merc you need proper roads and disciplined traffic. Otherwise its high costs of acquisition and maintenance seem to be unnecessary burdens on the owner.

With the exception of banks, telecoms and a few Indian pharmaceutical companies, almost all other Indian companies across the sectors have those inherent problems. So when they do not get tangible returns from their IT infrastructure they tend to pay substantially less for the same. That is the root cause of all the problems. The Indian IT companies focus mainly on the application development side with a distinct bias for manpower intensive projects. For them the earning per employee - or 'billing rate' in the industry jargon - takes precedence over everything else. Indian clients do not pay more than one-third of what an international client would pay. Add to that the inevitable delays associated with the domestic projects due to various reasons like unavailability of proper infrastructure, stiff resistance from the middle and lower management levels who see IT as a source of extra work load, erratic payment schedules etc.

Instead of staying away from such messy, low-profit (often loss making), difficult to manage projects the Indian IT companies - with the exception of Infy - have been systematically exploiting the Indian clients for training their freshers. The domestic projects are known as visa factories. Because of the strict regulations against import of cheap labour in most of the developed countries, visa / work permit applicants need to have a couple of years of experience as a pre-requisite. How do the fresh engineering graduates get that experience? Mostly through the domestic projects!

So how the MNCs are making money out of the Indian clients? Hardware supply is the key. They make enough money from hardware components - like storage devices, PCs, servers, routers etc. - to be able to cross-subsidise the application software. Indian clients - thanks to their great IT awareness - are very reluctant to pay for something intangible which they cannot touch and weigh! Software is such a component. Hardware, fortunately, does not suffer from that bias.

Before I end this post I want to share a funny but real story of an Indian client with the readers. Around ten years back one of my friends used to work as a free-lance developer. One of his many clients was a typical Lala-type trader. After making big money in trading, he wanted to add some class to his business. So he wanted my friend to develop a software for his company. After toiling for a couple of months and bearing with the paan-chewing, gaali-spewing Lala he finally delivered the software in the form of two floppy disks. The installation and the subsequent setup went off smoothly. The bill of twenty thousand gave the trader a big shock. "Bees rupiya ka floppy de ke aap bees hazar rupiya maang rahe hai? Sharm nahin aati?", he said. It proved to be rather difficult to convince him. My friend tried to educate him first. Then only could he hope to convince him! Finally he gave up. He settled for five thousand. Last couple of years the same friend of mine is in US. He is still a free-lancer. He never goes near any client whose skin colour is even vaguely brown.

BTW, don't take that last line too seriously, I am pretty sure he just forgot to add a smiley ;-)

Open Source Development - The Perfect is the Enemy of the Good

2007-03-27T21:05:00.000+05:30

Subversion developers Ben Collins Sussman and Brian Fitzpatrick in a talk titled How Open Source Projects Can Survive Poisonous People (And You Can Too) (Google Video) stress the need to avoid analysis paralysis . One of the ways paralysis is caused, explain the developers, is the attitude of perfectionists that makes them indulge in endless technical discussions on mailing lists - this includes seemingly interminable discussions on even minor details; The 'Painting the Bikeshed' analogy by Poul-Henning Kemp illustrates this point. As Ben and Fitz put it in the presentation, "The Perfect is the Enemy of the Good".

I wonder if that's the reason they have put up with WebDAV for so long ;-)

Dreaming in Code - Andy Hertzfeld Quotes

2007-02-18T14:19:00.000+05:30

There are some great quotes from Mac hacker, Andy Hertzfeld in Scott Rosenberg's book Dreaming in Code.

"My style is to get something going really quick and then turn it into the great thing that is the reason you're doing it. You're not working to have it be run-of-the-mill. You're working on it to do something great. But you need to get it started! The key is getting exciting work going; the rest of it will take care of itself. You're sparking off each other - a virtuous cycle - once you're doing the thing you're there to do."
"There's no such thing as a typical software project. Every project is different."
"And in this meeting, we kill the snake - we don't just make plans to kill the snake."
"I'm the kind of developer who likes to throw lightning rods around. To make a great program there's got to be at least one person at the center who is breathing life into it. In a ferocious way."
"My way of writing code is, you sculpt it, you get something as good as you can, and everything is subject to change, always, as you learn. But you climb this ladder of learning about your problem. Every problem's unique, so you have to learn about each problem, and you do something and get a better vantage point. And from that vantage point you can decide to throw it out."
"Code is cheap. But often it tells you what to do next."
"We all need more glory as designers - to show we can design another great thing. Everybody who has a first success, especially when it's young, wonders: Was it luck, or was it skill ? Well, it's a little of both. If you can do another really great one, it shows the world something."

Mary Lou Jepsen - Inspiring

2007-02-12T16:09:00.000+05:30

At age 29, Jepsen found herself suffering from blistering headaches, confined to a wheelchair, and sleeping 20 hours a day. She was just about to drop out of school when an MRI revealed a tumor on her pituitary, a small gland at the base of the brain central to hormone production. She underwent surgery to have the tumor removed and emerged from the ordeal ready to move on with her life. “There’s a stigma when you undergo brain surgery: are you still smart or not? So afterwards I tried to challenge myself to find out.” She finished her Ph.D. in the next six months and then cofounded MicroDisplay, a Fremont, Calif.–based company that manufactures liquid-crystal-on-silicon chips for high-definition TV displays. She left MicroDisplay in 2003, citing “creative differences” with its chief executive, but within days Intel was recruiting her.
Her health problems weren’t quite over, though. As a result of the operation, Jepsen’s body now makes no hormones, requiring a rigid schedule of twice-daily hormone supplements to keep her alive. Now that she’s a globe-trotting computer executive for the OLPC venture, the regimen can be tough to follow; last March she went into adrenal shock on board a plane, forcing it to make an emergency landing. (On the bright side, Jepsen reports that as a result of her hormone deficiency, she is unaffected by jet lag.)

That's Mary Lou Jepson, CTO of the 'One Laptop Per Child' (OLPC) project.

Writing vs. 'A Real Programming Job'

2007-02-12T15:26:00.000+05:30

Financially, writing science fiction is to writing technical books as writing technical books is to having a "real" programming job.

- Leonard Richardson, co-author of Ruby Cookbook and Beginning Python,
in an interview.

How Come ZeroConf is Not Yet Wildly Popular

2007-02-12T14:38:00.000+05:30

I have always wondered why Zeroconf adoption has been so poor. Stuart Cheshire of Apple who pioneered this technology, briefly explains one of the reasons in a presentation introducing zeroconf(6:16-7:30) at the 2006 Emerging Telephony Conference.

I imagine a number of people are thinking, well, if it's so great, how come I haven't heard of it ? And, that's an unfortunate history of marketing mistakes - I called it Zeroconf, but Steve Jobs thought that was a dull name. So he called it Rendezvous which is a really good name. But, some other company thought so too and they had registered the trademark.

Then we had a couple of Years of the marketing people and the lawyers doing their stuff. And meanwhile we had a technology without a name. So, in that two years every single printer vendor adopted it. But they didn't have a name to call it, which is why you won't see it on the box. But if you have a printer on your network, an ethernet printer, or a 802.11 wirless printer that you bought in the last couple of years, then it has bonjour in it. And, on the Mac you go File->Print and it's right there in the print dialog; No setup assistant, no wizard. On Windows we can't make it quite that easy. But you can download from www.apple.com/bonjour and just run the setup assistant and it will find all those printers on the network, and configure them for windows for you.

FOSS.in 2006 Notes

2006-11-29T20:26:00.000+05:30

Notes from some of the talks/discussions, that I attended at FOSS.in, held in Bangalore between 24-26'Nov 2006.

Suparna Bhattacharya, the Linux kernel contributor who works for IBM talked about how kernel hackers are obsessed with making the code simple and the minimalist spirit that pervaded the entire kernel development. She also drew parallels with Buckminster Fuller's idea of ephemeralization - doing progressively more with less. I have noticed that this obsession with internal simplicity and cleanliness is something that is noticeable much more with the better open source projects than in corporate development environments. While it's disconcerting in the beginning, after a while you wonder how development could have been done in any other way. Interestingly Suparna also mentioned that she had used a BBC Micro at school - this tallies with my theory that a lot of good programmers are folks who have had early exposure to computers.

Aaron Seigo, the KDE guy, talked about the planned features for KDE 4 . The Tapioca project aims to make VOIP integration with the linux desktop a reality; SVG support is maturing; The threadweaver project hopes to make writing multi-threaded apps easier; Project Strigi will make desktop search seamless and has a DBUS interface; does meta-data search as well. Aaron also mentioned working with HCI teams to make KDE more usable to regular users - I personally like this idea. The KDE project's UI ideas have always seemed juvenile to me.

Luke Kanies talked about Puppet - a server automation tool that comes with it's own declarative language. Puppet is written in Ruby and is the result of Luke's frustration with existing tools. 'Developers do not think about manageability', feels Luke. He also added that there are many opportunities to add small pieces of functionality. Puppet has a central server, a portable resource layer and agents on the network that do the actual automation work. The custom configuration language can be versioned and changes tracked easily, since it's plain text. The author of Puppet is very interested in building an open community around this GPL'd software and said that it might take many generations to make the tool great, and he would help create those generations. Resources in Puppet could be anything - Packages, Users, Groups - just about any low level system object. Resources can have multiple low-level providers. The platform specific details are abstracted away. Clients or Agents provide the server with facts about the hosts every thirty minutes or so. Puppet uses SSL everywhere. Though it's been around for only a little while, it is already in production in a couple of places in Europe and US. Stanford Univ. is a paying client. There is no windows version yet and I could not help feeling that the lack of a GUI might be a disincentive to most admins.

This is one area in which I have lots of personal interest, and my current job also involves similar work. I chatted with Luke a bit. I think it's high time we had a good cross platform (*nix, Linux, Mac, Windows)open source tool in this area. It's time to kick the pricey vendors out - they waste our time and money, and have no clue about real requirements. Good luck to Puppet and Luke Kanies. Google videos of his talks elsewhere.

PHP man Rasmus Lerdorf gave a great talk on how to build a good community site and optimizing sites to handle high traffic; even though he used PHP as the example, I thought it was quite general. Rasmus appeared to be a bit upset with the loads of criticism levelled at PHP these days and pointed out the people making negative blog posts about PHP do not realize that Wordpress was a PHP application. I am not sure if this is a good defense of PHP, the language has real problems. But I think it's helpful to remember that for a long time and even now, it's a great way to get things done quickly. As he himself put it, PHP is for weekend warriors. Rasmus actually began his talk by hypothesizing about why geeks are so motivated to work on open source software or even do non-programming community work. He mentioned the case of some users on Yahoo Answers who answer 500-1000 per day questions some times and attributes it to oxytocin, the hormone that gets released during human interaction. Even geeks who rarely see daylight and live in their mom's basement, have the need to connect with other humans, and hence the motivation to take part in net communities. Rasmus feels that it's not what people think about you, but rather what they think about themselves that matter.

Rasmus observed that people should be able to improve a community site by catering to their self interest;he mentioned that no worthwhile community can be built without giving up ownership - if you don't give up control, you will never build a decent community of users. I personally think this works for some categories of software/communities and not for all. Dan Bernstein, the author of Qmail maintains total control over the direction of the project, Napster was Shawn Fanning mainly, and I could go on and these were projects that were so useful that the community ignored the idiosyncrasies of the owner. PHP is now mainly community driven, and became so around 1997; Rasmus' code in PHP is less than ten percent now.

He took apart a small PHP app with the help of Valgrind and showed how profiling an application can be counter intuitive. It was extremely interesting to hear this portion of the talk and I was unaware that valgrind could be put to such use; http_load was used to load test the site. He also mentioned the XML support in PHP5 including Simple XML, Yahoo'd geocoding api, and cautioned against using participation gimmicks like paid blogging. Solve one problem, a clean and intuitive UI were other tips from Ramus. Surprisingly he said that he finds programming to be boring and tedious, and his main interest is in solving problems. Slides here.

Christophe Hellwig, a linux kernel hacker tried to motivate contributions to the linux kernel. Among other things I learnt that one doesn't have to sign over copyrights when submitting code to the Linxu kernel. In contrast contributions to FSF projects need to sign over copyrights to the FSF and do some paper work as well. Christophe said that for result-oriented people there are lots of interesting problems to solve. Lots of modules need help. He explained how corporate contributors to the Kernel were often at conflict with the kernel developers due to pressures from the boss. There was a Sardarji in the audience who asked how many years it generally took to become a kernel developer - the audience laughed for no apparent reason ;-)

Andrew Cowie made an energetic presentation about the structural problems facing open source projects. He outlined at length the problems with centralized version control, build systems, bug tracking systems, and how they all combined to make collaboration between projects difficult. He made the very interesting observation about the committer/non-committer divide created by centralized version control systems. I wonder if DVCSs really solve the problem - you could have your own repository, but what if your changes are never merged into the main repository - C'mon stop pretending that there will be no 'main' repository with distributed version control systems. Besides the problem is how do you discover these repositories - DVCSs do not address the question of discovery at all.

The penultimate day of the conference saw a panel discussion by Sudhar 'Thaths' Chandrashekar; the panelists included Atul Chitnis, Kishore Bhargava, Karunkar Guntapalli, Arun Sharma, Frederick Noronha, KDE developer Sirtaj Singh Kang,and another bloke whose name I didn't get. Atul Chitnis who incidentally does not have too much by way of actual contribution to FOSS projects was predictably the most vocal about young people these days, yada, yada and the death of curiosity. He even made some vague connection between the ability to take apart devices and the ability to hack. In fact there is no such connection. Check out Karl Fogel's excellent article . Only in India do such vague assertions fly without fear of contradiction - it is no better than corporate software environments where the 'get things done' types and 'Excel gymnasts' are regarded as being more useful than good programmers. One recent college graduate finally got fed up with 'oh-young-people-these-days' rants and explained some interesting work they had done at college, that helped reduced the cost of the implementation. He explained that they customised the software extensively and the proprietary choice was very expensive. He also felt that Rs.500/- was a little high for college students. Instead of patiently explaining to a newbie, Atul lit on to this poor chap. One of the panelists was very honest and admitted that he had failed in his MCA exams, unlike lots of pretentious desi FOSS folks who are still enamoured of pedigree. The questions were all predictable, but the answers were better even though the pessimistic tone was a tad too much for me. I can't figure out how Atul arrived at the conclusion that things were going downhill - In fact there have never been so many Indian contributors as now. Atul and co seem to think that being well informed and following every single mailing list out there, and latching on to every passing tech fad is what contribution is all about. Well Atul, with my limited experience in open source projects I can tell you that working on nontrivial patches is real work - the kind of work that requires you to put down your head and do real hard work. Next time you feel like making a claim like this, please do some honest research and go through the commit mailing lists and actual dev lists. I know that you are playing to the gallery and pretending to be the sole saviour of open source in India. Just as life is what happens when you are planning, real open source contribution is what happens when non-programmers who are over the hill are talking about the glorious days of tinkering with modems. Honestly I don't know what the OSS contributions of the panelists other than Sirtaj, and Karunakar are.

It was funny that while one the very visible participants who is a Google employee did not miss an opportunity to let everyone know that he was working with the world's largest organized cult, I heard from another participant at the conference that a very respected open source contributor and O'Reilly author had resigned from Google after spending just three months there. A few days back news of another very respected oss contributor quitting Google trickled in. I know this guy and he is not the one to go around advertising his coolie status even if he works for google. Integrity and self respect count for more than grovelling before the benevolent master.

I attended this event after six years. The first time I met RMS. Nat Friedman's behaviour during his presentation was extraordinarily juvenile - he used the F-word so many times during the presentation that I got irritated. I didn't get a good impression and the many twists and turns of the GNOME project since then have only confirmed the fact that it is run by a bunch of juvenile delinquents. The recent Novell-Microsoft fiasco only serves to confirm my initial impressions.

Best comment/observation of FOSS.in 2006
When a member of the audience, strangely enough a Novell employee, asked if it was not possible to enjoy the freedom without getting into license wars, someone from the back (either Luke Kanies or Andrew Cowie) shouted:

That's a fan club - not an open source software project.

Indian Code contribution to FLOSS

2006-01-01T20:51:00.000+05:30

There have been some discussions about why there is so little code contribution from Indian programmers to free software/open source projects following the recently concluded FOSS.in conference. Linux Weekly news has some interesting observations to make:

Even if only 10% of those attendees were truly active in kernel development, one would expect to see a significant amount of code from Bangalore working its way into the mainline kernel. And there are some Bangalore-based kernel hackers who are active on the mailing lists and who are contributing code. But their numbers are far smaller than one would expect after seeing how many people are interested and knowledgeable in this area. India is, as one developer put it, "the world's biggest consumer of free software," but it is not a huge contributor.

The usual suspect, the much maligned (and rightly so) educational system in India:

By several accounts, the problem starts with the university system. The Indian universities are strongly oriented toward the creation of employable graduates in large numbers; a number of FOSS.IN attendees described them as "assembly line" operations. There is a strong emphasis on passing tests and getting through the system on schedule, and, it seems, little interest in encouraging creativity and curiosity in the students. The universities were described as a conformist environment with little love of those who have their own ideas of how things should be done. The end result, as expressed to your editor, is that most students have had any love of hacking beaten out of them by the time they graduate.

The fact that the universities are, for the most part, hostile to Linux and free software does not help either.

The problems with the typical corporate developer in India are outlined:

Neeti's talk described Indian developers as needing to have their jobs laid out to them in great detail. They want to know where their boundaries are, and are uncomfortable if left to determine their own priorities and approaches. Your editor's initial reaction was that this claim sounded like classic talk from a pointy-haired boss who does not trust his employees to make decisions. Subsequent discussions backed up Neeti's claims, however. A few Indians told me that Indian employees require a high degree of supervision; perhaps that is why the pizza stand at the site required two-levels of necktie-wearing bosses who apparently did little to actually get pizza into the hands of conference attendees. It is not that Indians lack the intelligence to function without a boss breathing down their neck - that is clearly not the case - but all of their training tells them to work in that way.

So if one were to construct a stereotypical picture of an Indian software developer, it would depict a person who sees programming very much as a job, and not as an activity which can be interesting or rewarding in its own right. This developer is most interested in getting - and keeping - a stable job in a country where an engineering career can be a ticket to a relatively comfortable middle-class existence. Keeping that job requires keeping management - and coworkers - happy, and not rocking the boat.

There is also speculation about potential problems caused to a corporate developer due to participation in an open source project:

For such a developer, the free software community is not a particularly attractive or welcoming place. A developer who contributes to a free software project may earn a strong reputation in the community, but that reputation is not appreciated by that developer's employer or co-workers, and is not helpful for his or her career. Criticism from the community - even routine criticism of a patch by people who appreciate the developer's contributions in general - can be hurtful to a career in a culture where open criticism is not the normal way of doing things. Developers who expect to have their job parameters laid out to them in detail may feel lost in a project where they are expected to find something useful to do, and push it forward themselves. And these developers, while being possibly quite skilled in what they do, often have no real passion for programming, and leave it all behind when they leave the office each day.

LWN's take on the issue is mostly correct. I am stunned at how close the article has come close to diagnosing the problem, and if all of it is from a one week visit to Bangalore to attend foss.in, it's the all the more impressive. But there is very little input from developers who have actually made headway in sending patches and becoming committers in FLOSS projects and that makes the article a little bit shallow, as also it's failure to consider socio-economic reasons. Regarding the latter, I think the best explanation comes from Thaths from mailing list thread:

Based on a sampling of my White geek friends, I would say they are predominantly children of white collar baby boomers. Many of them were exposed to computers their dads brought home from work at an early age. I think similar conditions exist in India today for the kids of today.

I came to the same conclusion a couple of years back and have noticed that most of the American hackers whom I have read about, or have worked with, have had very early exposure to computers and programming and their comfort level is consequently very high. I am starting to notice that a bit with the generation born in the 80's and 90's in India as well. They tinker around with hardware with relative ease, play more games, and seem to know their way around computers much better than their parents, who incidentally slogged it out to create this improved environment for them. However I think even the socio-economic reason is only partial, and no one really knows. There is a real need to put on a web page the Indian code contributions and collect the experiences of Indian hackers.

Marc Canter on Chandler

2006-01-01T20:39:00.000+05:30

Marc Canter says about Chandler:

Three years later and at least $5M Chandler hits version .6. Man oh man what we could do with that sort of money!

The Best Program according to Don Knuth

2005-10-11T20:11:00.000+05:30

The Nerd TV interview with Andy Hertzfeld contains another interesting bit. According to Don Knuth, MacPaint was possibly the best program ever written:

I have another anecdote that has to do with Open Source and the Macintosh. In January, 2004, the Computer History Museum had a little presentation about the marketing of the Macintosh with all the original Macintosh marketing team up on the stage. And a large percentage of the rest of the Mac team in the audience. They had a Q&A toward the end and an older guy got up and said he thought MacPaint was probably the best program ever written. Was it possible for him to see the source code? It turns out the person asking the question was Don Knuth - another one of my heroes. He came up to me afterwards.

Andy Hertzfeld on GNOME

2005-10-11T19:51:00.000+05:30

I remember buying the Eazel T-shirt to show my appreciation for the folks who had shown the earliest signs of good taste in the free desktop world, back when I thought that the Linux desktop was going to make a huge impact. Raph's SVG work which resulted in the first SVG icons on the desktop were pretty cool, Nautilus generally seemed to be a notch above the rest as far as polish and attention to detail were concerned. Then came the crash and Eazel dissapeared. In an interview with Nerd TV Andy Hertzfeld explains what they set out to do and how things have actually turned out with GNOME.

We were just building on top of Gnome. Our goal was to make Linux easier to use and part of that is work on the GUI side of things and part of that was work on system management. We evaluated the alternatives and ended up picking Gnome. We built something called Nautilus, a new file manager. I never thought of it as a file manager, I'd call it a graphical shell. We got about halfway done what we wanted to. It's just disappointing we never got to take it as far as we wanted.

It is still used. If you install Gnome on your machine you'll be installing Nautilus. It has drifted away… it didn't live up to its potential. It still has some good stuff in it, but he people who took it over did not have much forward thinking vision. They ended up, "when all else fails just copy Windows." So it has kind of drifted into that modality.

The 37 Signals Way

2005-10-11T16:32:00.000+05:30

I remember 37 signals vaguely as the company that the Ruby on Rails guy works for and they seem to generate quite a buzz from time to time with bloggers, for reasons that I have never cared to investigate. That is until I listened to a podcast - "Lessons learned while building BaseCamp" by Jason Fried, the founder of 37 signals recorded at the O'Reilly emerging technology conference earlier this year. What blew me away was the refreshing approach to software development these guys have and how cogently it was articulated. While Jason Fried himself admits inspiration from Agile programming, I think it deserves attention on it's own merits, and for being able to set out a bunch of actionable items. Putting a new spin something that many people might know or might be obvious to others is not without merits, there is always value in re-packaging good 'old' ideas.

The semi-transcription/notes from the talk:

The talk revolves around four main themes :

Reducing Mass
Embrace Constraints
Getting Real
Managing Debt

Reducing Mass
Mass could be people, high overheads, long term contracts etc. It's difficult to change with more mass, and being able to change is extremely important for any product development process. Find the right people -we look for people with certain level of skills. But, beyond a certain point it is not about skills alone, it's not about how many years experience you have with PhotoShop, it's not about bullet points. You need to work with people that are positive most of the time. In a small team of four one disgruntled person can bring down the whole team. Morale is incredibly important when building a product.You have to have people who are excited about what they are doing and are positive about what they are doing. Jason advocates hiring people who are well rounded, and can do multiple things. Don't hire someone who is just a programmer, or an information architect, or a designer, or a marketing person, especially on small teams. Hire people who can learn quickly new things (like AJAX) and apply it quickly.Hire Trustworthy people - trustworthy does not mean people who lock the doors at nights and remember to turn off the lights. It's very important that you don't have to cleanup after that person. The person you want to hire has to be a good writer - A programmer, or a designer, or a marketing person all have to be good writers, because of how people communicate with each other to each other these days - they don't talk that much to each other these days, they IM/email each other all the time. You have to find people who can communicate by writing. Jason's bottom line for hiring people - I will take someone who is happy and average, rather than someone who is a Guru, who is the best in the business, but is disgruntled and frustrated. Such people will bring down the team in the long run.

Constraints
Once you have put in the team in place, you have put in the constraints. Most companies instead of embracing constraints start thinking, we don't have enough money - let's raise more money, we don't have enough people - let's hire more people, and so on. Don't do that! Look at your problems and come up with creative solutions. Embrace those problems, and don't push them under the rug. Some of the constraints that 37 signals had to deal with when building BaseCamp:-

Prior commitments: 37 signals was still a web design shop and initially BaseCamp was about 20% of the time spent. David Hanson the only programmer on the project was still in school and was committed to some of his own free lance work as well. We didn't put everything down and say 'let's focus on BaseCamp'. We made sure that the time spent on base camp is valuable time. The time difference between Chicago and Copenhagen (where Mr.Ruby on Rails lives) is seven hours -On paper it looks like disaster - we decided it was a good thing, you communicate mostly through IM and email. When you talk through IM, you have to be really specific. You chunk things into small pieces and you move on. We are self-funded. Spending money should be the hardest thing that you ever do. Make sure that every penny that you spend is a hard penny to spend. That will ensure that you end up making much better decisions and you spend money in better places. And finally having a small team - we had a team of three people, one programmer, and two designers. People in smaller teams have more power, you can't hide behind twenty other people.Everyone has to do something. It ensures that everyone cares about the product. These were constraints that were put upon us and there are ones that you can put on yourself. Building less software - that's the constraint that we decided to impose on ourselves. It's not about building feature-by-feature, but less features intentionally. When you have less software, you lower the cost of change.

Managing the opportunity for things to go wrong is the key thing. Less support is required. Technical support will kick your ass - It will destroy you. We don't advocate hiring a bunch of people to do technical support. You want to build something that is manageable. Outsourcing tech support is a bad thing. Finally, you also want to encourage human solutions. Less software, less features means less rigidity which means that people have to come up with their own way of dealing with the product. We all hate the paper clip guy in the corner - you should give people enough to solve their problems in their own way and then get out of the way.Ta-da list is an example - a really simple to-do manager.You look at the patterns in the users behavior and maybe there is an opportunity there to improve the product based on how people use it, based on real product use, not fake paper mock-ups etc. Look for usage patterns in a real environment and add a feature if it turns out to be necessary or if people ask for it. It's always better to start with less than more. The idea of building less software leads to the next idea - how do you do releases - You release half-a-product, not a half-assed product. There should be no scope creep, only scope reduction.Everyone is doing a beta these days and that's ridiculous, Everything is a work in progress and continue to improve it , have confidence in your product.

How to do scope reduction
Say No, by default, to all new features. You want to listen to the people who are using it. Your customer base will tell when you need to add something. If you get ten requests, then it's a "maybe", if you get twenty then it is stronger (demand), thirty requests - then probably it is something you need to add. Listen to your product, and listen to your customers. Say No by default, see where the gaps are, observe the usage patterns before deciding to add a feature. You will build a better product based on real data.
You want to ignore details early on - you can spend a week on something that doesn't really matter. A lot of small things that you think matter don't really matter. Take care of the big picture first. Fill in the details close to the release. If you have time before the release make existing stuff better instead of adding new features. The idea that every minute should be spent on a new feature is a bad idea. Focus on what you have - make it rock solid, make it bullet proof, make it great. There is always room to add stuff later.

All decisions are temporary. You should be able to change all decisions that you make, in theory - realistically you should be able to change almost all. You can change decisions if you have less mass. You want to make decisions that you can change, every decision should be temporary. You should should take decisions just in time - You should make decisions when you have real information to take a decision.

Make mistakes in the open, admit it to customers and fix it quickly. Jason admits that this is not applicable to every kind of project. It applies to web based software, especially the UI portion. Make small changes, iterate often.

Getting Real
We don't do functional specs, we don't do diagrams, we don't spend a lot of time doing use cases, we don't spend a lot of time doing things the customer will never see or never use. We want to be in front of the customer experience close to 100% of the time.If it takes four months to make a product I want to spend 3.9 months in front of the actual product.I don't want to slap user experience at the end. That's what happens when you are talking about these functional specification documents - functional specs are political documents.They are documents used to blame one another - it was supposed to work this way,it doesn't etc.It's also an illusion of agreement. Someone will say inevitably - "that's not what I intended".Well you signed off on it and politics starts. The design is the functional specification - programming needs to follow the user interface. When you are in front of your interface all the time, it is almost like your version 1.0 is someone else's version 2.0, because you are constantly tweaking real user experience that is not based on paper prototypes - people always have something to offer when they see the real thing. You are postponing those inputs by writing functional specs. Build simple things for all users including expert users.

Managing Debt
Manager your debt - If you have earned debt through some ugly hacks, pay it off as soon as possible by setting the code right. Don't get stuck in hardware or software that you can't change.

Making most decisions just in time
Japaneses factories that have just what they need, use new or better parts. Just in Time systems allow people to make better products. Scalability is a myth. If you are a yahoo or a google, then you really need to worry about it. If something is slow for 2 weeks, it is slow for 2 weeks. It's OK to make mistakes in public. BaseCamp was running on a single server and was being used by thousands and thousands of users, till about three months back. Then we switched to 2 boxes and we are in the process of switching to eight. If you invest too much money and effort up front setting up a cluster and you don't need that for 12 months you have wasted an awful lot of money and you could have spent the time on something more valuable. Admin interfaces are another - don't spend time on writing stuff that will generate extensive reports and what not. Actual customer experience first. BaseCamp is free for 30 days. We couldn't bill anyone in the beginning;figured it out after a while. Make decisions when you have the real information, not before.

Feeling the Hurt
The people who build the product should do the tech support on the product for as long as possible. Jason still doing tech support and he owns the company. I want to know what's right, what's wrong. 37 Signals get only forty emails a day - simple product that is well designed. Don't want to get the same emails over and over again. Chefs have to be waiters in certain restaurants.

Publicity Amplifiers
Here are some publicity amplifiers - for getting the word out about your product. With no marketing budget,no PR budget,no money, how to get the word out ? Lace the product with feature food - features that users use and spit out small little features and people tell others about it. Small little features that users eat and spit out all over the place. In BaseCamp you can subscribe to your feature milestones via Ical. We talked to mac sites about it - this news blurb generated tons of traffic and tons of sales. Same feature is available via RSS as well - the blog world loves RSS. The Yellow Fade technique generated about 325 references on google. This small technique generated more buzz than the product itself. Hold back some features, and do it 30 days after launch. It will show that there is momentum happening. people will really love that and it shows your commitment to the product - You didn't build a product only to move on to something else.

Transparency and Trust
If you make a mistake, tell people.We made a mistake and had to be manually in front of our servers for forty hours. No data was lost and we had to re-start the server constantly. We could have hidden that;but, we posted a long explanation - lot of people trust us now.

Blogging
Blog as much as possible about your product. Google will send lot of traffic your way.

Question from the audience:
How do you work with David Hanson (who is in Denmark) ?
Jason: We use BaseCamp to do 8-10 hr iterations in order of priority - we don't worry about what will happen eight months down the road.

Looks like Ruby on Rails is not the only thing to come out a small software startup with less than half a dozen employees.

Bruce Eckel on Consulting

2005-08-26T17:48:00.000+05:30

Great post from Bruce Eckel on consulting. His definition:

"I think consulting is when you have some kind of special expertise -- come by through hard struggle and learning -- that you transfer to a group of people, in a relatively short period of time, and in a way that is unique for that group. I also think that consulting involves addressing particular issues faced by that group. "

He also addresses a very common industry phenomenon:

"It's tempting to make the analogy to a ponzi scheme, but it's not that bad. Both forms of a consulting firm (first-class consultants only vs. high-tech body shop) have value. The problem is the temptation to present the second form as the first, in order to apply the same high fees of the first-class consultant to each additional body added to the shop. The goal of the company becomes "how do we transfer the aura of authority from the high-image consultant(s) to anyone who works for us, so that we can charge the highest fees possible?" Or to simplify, at some point the bean-counter mentality takes over and the mission statement of the company goes from "how do we provide the greatest value to the customer?" to "how do we charge the highest fees possible?" (You can argue that this is the fundamental shift that any publicly-held company goes through. After all, a public company is legally beholden to maximize shareholder value, so how could it be otherwise?)"

I have noticed this myself, when I was working in Boston during the tail portion of the dot-com years. I was chatting to an Indian body shopper who was a sub-contractor for Andersen Consulting; While Andersen made $450 plus per hour, this sub-contractor got about $70 - You can imagine how much would have trickled down ultimately into the pockets of the programmers, who were paraded as Andersen consultants to clients.

Another nugget from the article:

But advice that appears to be rational does not necessarily solve real design problems, as we saw in the "structured design" movement of the 70's. This seems to happen over and over in our industry -- it's easy to focus on one aspect that seems to be "the solution" and miss the big picture, and to produce unintended consequences that eliminate the benefit of what may seem so clear in isolation.

Slash Wisdom

2005-08-26T17:10:00.000+05:30

From an "Ask Slashdot" posting titled "Uneducated IT Managers, and How to Deal ?", I found this gem posted by Pope Benedict XVI ;-) :

"It's just about impossible to find a job working for someone whom you respect. "

To me, that about summarizes the crux of the problem with working in the IT industry. But strangely this nugget of wisdom has been marked as funny, instead of insightful, or some such. Setting the filter at four and reading reveals more insightful comments. Slashdot, for it's flaws, sometimes throws up insights that knock out your breath momentarily, or better still has the reader in splits - Some of the trolls are outrageously funny without being crude or offensive.

Wisdom of the commons, or shall we say, Slash Wisdom !

Joel on Software Writing

2005-08-08T00:26:00.000+05:30

A few interesting points ( mostly paraphrased) from a forty minute interview with Joel Spolsky on software writing:

Most software writing is awful. There is a high correlation between successful software and writing.
Programmers are used to writing code for compilers; compilers don't care about order. Humans comprehend stories - can't give them a bunch of definitions and tie it together. Most programmers, who write for compilers , don't know how to write for human brains.
There is an awful lot of writing there about technology, even internal functional specifications that just doesn't get read, because regular people don't understand it.
At Fog Creek software (Joel's company) they ask for writing samples, not necessarily technical stuff from all candidates. Because we (Fog Creek) really do find that the ability to communicate clearly and effectively, is the crucial difference between an acceptable programmer and a great programmer that can work on teams, and get their ideas understood by the world.
Once you make software that mediates between humans, then the design decisions in the software have anthropological implications. In Joel's latest book, Danah Boyd talks about some websites that require a strict social relations - such requirements are pathological, not normal. People are actually creating software and websites that require you to conform to a set of behaviors that in the real world would get you incarcerated.
Paul Graham's assertions about great hackers - they use open source, they don't use Java etc flies in the face of facts. But his essays are beautifully written. Joel said his hands start shaking whenever there is a new post on Paul Graham's site.
Phil Windley asks an interesting question about whether it is opninions that make software writing interesting. Joel says that's where the value is being added (in opinions) and goes on to add that with great respect to the American journalistic tradition of objectivity, the only thing it results in is lot of cnet's news.com, that is very bland, that reproduces a lot of press releases , or they get someone from Sun to say something about Microsoft's announcement.
According to Joel, the number one rule of technical writing is, "Show, don't tell". Technical writers violate that rule so often. So much technical writing reads like mathematical proof - all the information is here, if you can decipher it, you can understand it. That kind of stuff doesn't get read unless it has to get read. The stuff that people read are the ones with stories in it. Human beings are story tellers. We have been telling stories around camp fires for hundreds of thousands of years - that's what out brains are good at and that's we enjoy, that's why TV is popular and that's how messages get into our head.
When people started using PC's there was lot of piracy, lot of the books were just shovelware. Companies were trying to make the books as fat as possible. At one time software could not be used without books. It was not until the GUI generation and much later, that software companies realized that they don't have to produce the manuals at all. These days some of the best books are the ones written to explain the domain in which software is used. Quick Books' manuals, for example, teach you a lot about the basics of book keeping.
Amazon ratings are not accurate and are not done by professional people ( I personally disagree with this sweeping characterization). Joel said that there were folks with axes to grind who wrote reviews on Amazon and gave the example of a Microsoft techie, whose book got bad ratings, because he used to be aggressive in some news groups.
If you are a computer programmer, put yourself in another person's mind - what do they know at any given time ? Try to answer the questions in their minds in the order in which they are having it. If you write a document the reader first with what the document is about, and start answering the questions that they might have in their minds.
Some of the great functional specs are written like a story.
Blogging is a good way to exercise the writing muscle. Joel noted that he had gotten a bit over defensive in his writing of late.

RPC

2005-08-06T21:40:00.000+05:30

Your brain does not use RPC to talk to your pancreas.
-Werner Vogels, CTO, Amazon.com

The talk titled E-commerce at the interplanetary scale is good, but a lot of it went over my head, especially the stuff about Epidemics; but he did admit that he was a recovering academic ;-) .

Miguel de Icaza on on J2EE domination

2005-07-29T01:45:00.000+05:30

Miguel explains in an interview why J2EE has such a stranglehold in the application server market:

There is a segmentation of the application server space, which is the high-end market, the mid-market, and the low market. The low market is anything that costs less than $200,000 to develop and deploy. It includes every technology that you’ve heard of. Then you’ve got the high market, which is any project where the cost of deployment is over two million dollars, and in that market J2EE is firmly entrenched. There is no other technology considered today in that application space. In part that’s because people need to have multiple vendors providing the same solutions, so they like the fact that there’s BEA and there’s IBM and there’s Sun. There’s different J2EE providers. There’s also different hardware providers. So that market is very hard for Microsoft to penetrate.
"That leaves the mid-market, which today is about 50% Java, with the other 50% made up of ASP.NET, and a couple of other proprietary frameworks."

Adam Bosworth on a new data model for the web

2005-07-24T18:37:00.000+05:30

Following the excellent presentation by Michael Tiemann at the MySQL user's conference organized by O'Reilly media, itconversations has posted another great presentation - this time by Adam Bosworth at the same conference. The talk titled "Database requirements in the age of scalable services" and Adam Bosworth explains his vision to make data access on the web simple and standards based, and why he thinks RSS 2.0/Atom are the beginning of the unfolding of that vision.

The presentation starts with a joke about 'Microsoft Project' - Bosworth says he is amused to hear about people using Microsoft Project and how it is Microsoft's secret weapon to stop everyone else from competing. Some of the points I noted down while listening to the podcast.

Everything looked irrelevant to AB after he got excited about the web.
How did the web happen ? Tim Berners Lee hit a perfect storm of productivity and using HTML, HTTP any 'P' (Perl, Python, PHP) programmer could generate content. Simple was huge and everybody could play with HTML and HTTP. HTML is sloppy - everything is rendered
without a complaint. It takes a licking and keeps on ticking. You don't have to be the high priest of syntax. But that's not the case with XHTML - and that's not a good user experience. Web pages could read/edited on all operating systems
Databases are not good at partitioning - partitioning of data is very important to scalability. But the web does a good job of this, by distributing data. The web is also good at caching - At google they
observed 120,000 hits per second on a certain blog and the only reason the infrastructure didn't melt down was due to the amount of caching done on the web by proxies, front ends and google's own front end. Statelessness - the coarse grained interaction is also another reason for the scalability of the web. Clients talk to servers in terms of chunks of data: go to data when you are ready, not
continuously.
Google combines lot of simple minded techniques with brute force to deliver. Google has lots of Ph.D's and everyone is a General Patton driving tanks. Take the spell check that google does when you type in an incorrect (or sometimes correct) search string - it's based on the very simple technique of tracking failed searches and what users type in after a failed search. This kind of brute force enables google togo through petabytes of data in seconds.
Once you start to search, this whole business of putting things in folders begins to diminish in importance. It's very hard with folders to remember where you put what. Folders are not efficient, searchesare.
The Vision is to take the database and do the same thing for the web (as was done for content). Can we take all the info on the web andmake it easily findable ? Now you get content, not information.
Need something that scales massively and linearly. Originally thought that we would do it using XML. We created a tower of Babel (with XML) - websites need to support only one grammar with HTML. A working group took four years to come up with a spec for a XML Query standard. It's
better to spend six months and learn the rest from customers. The query standard was not simple like the web - the schemas were very complicated. AB also found the WS specs to be very complicated. Why did this happen ? The companies that came up with these standards were big and were trying to protect themselves. They were people at companies like IBM and MS. Frankly, they were trying to make it deliberately hard
AB apparently is a technical advisor to MySQL and had some advice for them as well: Basically MySQL is trying to be Oracle by adding support for procedures, triggers, and views. All of this is about centralizing processing logic in the database. To be blunt centralizing processing logic in the database is a bad idea - doesn't scale. Centralizing logic doesn't give you scale. Advice to MySQL - don't do something because you want to be Oracle. Because Oracle isn't big enough andcan't deliver on billions of queries.
We need an Open model for data. What's not open today is how you talk to a database - the actual wire format. There is nothing like HTML/HTTP for data. This is a very 20th century way of thinking. Open up wire formats to serve any kind of information - this will bring enormous changes to computing centered around data. Need open standards for different types of items with one single grammar. It will have to be sloppy. Open up and democratize the way data isserved.
Big believer in stupidity - virtues of dumbness.
We are actually starting to realize this vision - RSS2.0/Atom are going to be for data what HTML was for content. They are going to be the Lingua Franca of consuming data. Surprisingly simple and sloppy. These guys got the web and that's why it is catching on like wild fire. Atom was formed by a consortium of bloggers and the two formats areisomorphic.
Data queries have to be such that they don't need data spread across machines - if a query uses data from four machines, it isn't going to work very well. Queries need to run at an item level - it's not
technically as complex as sql
AB made it emphatically clear that he was not talking about the semantic web and called RDF an empirical failure. RSS 1.0 had an RDF grammar, RSS 2.0 doesn't have an RDF based grammar. Ordinary programmers do not understand how to model something as arcs, nodes,and graphs

Update: Looks like the O'Reilly network and ONLamp.com have also covered Bosworth's remarkable speech.

Michael Tiemann on Open Source

2005-07-23T00:07:00.000+05:30

The peerless ITConversations.com features a talk by Michael Tiemann of RedHat, formerly of Cygnus and the guy who first wrote the GNU C++ compiler. Some interesting points from the talk:

In 1995, when MySql started (this was a talk at the 10th anniversary MySql conference), Tiemann had already turned over the reins of the company to a president at Cygnus; the company was then making $6 million in revenue and employed more than sixty people, disproving the fact that you cannot do business with open source. Larry McVoy was kicked out of Sun in the same year for advocating open source as a strategy.
McVoy brought to Tiemann's notice a small company in North Carolina called Redhat - Tiemann advice to acquire a stake in that company was ignored and five years later, Redhat bought Cygnus.
It took him two years to figure out how to do business with open source.
Tiemann addressed the question of why OSS had succeeded and drew an analogy with the observation made by Alexis de Tocqueville in 1830s, that the sum of all individual undertakings in America exceeded the efforts of the government.
Disruptive technology always comes from unexpected quarters.
Cost cutting is not always the most interesting question - there is always more upside to inreasing revenue than to cutting costs. There are limits to cost cutting, while there are none to increasing revenues.
The real value of architecture is when something comes along and it can be accomodated quickly. Explained the value of strategy with an example from a Wall St. firm - a strategy that was 10-20% different resulted in a ten-fold increase in growth.
According to research done at CMU, the Apache project has ten to fifteen people who do eighty five percent of the work. But the total number of people who contribute is closer to 400 ( 388 ?). It's this extra non-core contributions that add lot of polish and quality to the software. Tiemann asked a rhetorical question about going to a venture capitalist to ask for money to recruit all 400 developers. (I thought this was a particularly effective illustration of the role of a large pool of contributors - more about this aspect in a future post, if time permits)
Quoted Eric Von Hippel, faculty at MIT and the author of Democratizing Innovation, on the role of user participation in design. EVH in his book explains in his book how Jack Welch managed to make GE a leader in plastics by bringing in the concept of user design toolkits. Previously GE's business model made it possible to get into selected market segments, in fact the largest customers. Jack Welch said that GE doesn't need to be the exclusive designer of plastics - Why can't customers design their own plastics ? This idea of moving of moving the locus of design from the supplier to the customer resulted in 85% of the designs coming from the customers ultimately. MT then asked "What rational person would let go of this opportunity ?"
Drew some analogies between disruptive open source technologies and themes from the book "Guns, Germs, and Steel" by Jared Steel. (The analogies about the conquest of the native american population by the Spanish seemed out of place to me)
He recalled reading Stallman's code for the first time - first time he said, he saw good code. The architecture and implementation unfolded in one view.
He remarked that even the newly elected board of the OSI did not have a clue about the reasons for the success of open source. That's the territory that needs to be protected and grown. He urged the audience and sophisticated users to think about it.
Over 70% of all projects on sf.net are covered by the GPL or BSD style license.
Quoted designer Bruce Mau about the fact that design is not very visible unless there is a failure and used Micrsoft's out-of-control spending on security to conclude that the shared shared model was a design failure. Microsoft's spending on security in Jul 2002 was $100 million and in Mar'2005 $2 billion. At this rate they should end up spending the rest of their bank balance on security. MT contrasted this with the improvements in open source software. The Fuzz report noted a 20% failure rate in Unix utilities in 1985. In 1990, by which time the GNU tools had appeared, the failure rate was down to 25% of that of Unix tools. In the same period the Unix utilties/tools had not improved substantially. Later the failure rate of GNU tools went down to 2% and by 1996 the GNU tools were a 100% clean.
Software patents are the lead banner for massive stasis - it really means that innovation will stop for the next twenty years.
Referred to Bruce Mau's work which rejects the notion of a client-designer relationship. The notion of an open source license may be obsolete, when it tries to break the above law.
How to enage the plurality of potential contributions in OSS.

A recommended addition to your IPod/whatever. A very thought provoking lecture from a person who has written a compiler, founded a company, been a CTO - a man of many hats.

Update: Looks like Michael Tiemann gave a similar talk (pdf) at the Redhat summit.

Dave Winer on working groups

2005-07-05T12:51:00.000+05:30

Dave Winer in a podcast explaining the evolution of RSS makes a interesting remark about working groups:

What happens when you put together a working group is that, you get people who like to be on working groups, not people who like to write software. And they have very different interests. They like to fly places, they like to have meetings, they like to discuss things, and discuss them again and again and again. They really like the discussions. They don't live to make the software. They want to make the discussions.

Links to articles in Joel Spolsky's book

2005-06-27T20:59:00.000+05:30

Kiran Jonnalagadda has listed the links to the online versions of the articles that form the chapters in Joel Spolsky's book , The Best Software Writing.

I am a bit dissapointed though - it's nowwhere near Joel's other book that is based mainly on his personal experience as a developer and manager at Microsoft and other companies.

Tyranny of the degreed and pedigree'd

2005-01-05T20:44:00.000+05:30

Sriram Krishnan lets off some steam about grades and getting a job in software companies. I have to agree with him and also sadly note that this particular article by Joel is not as good as many others that he has written. The desi obsession with grades, engineering degrees, and pedigree (IITs, RECs, etc) has now reached a point where it not only irritates and frustrates the truly bright(but who are not degreed and pedigreed folks), but is positively causing harm to the health of software companies.

Grades especially cannot be an indicator of problem solving ability or creativity in India! Why ? Because there is no examination system that the human mind can conceive of ,that, cannot be beaten by desis. Most institutions have a learn-by-rote learning system that squeezes out whatever little creativity and enthusiasm a young person entering college might have. More than any general arguments like these, I have encountered too many cases throughout school, college, and work, of friends and colleagues who have high grades or went to brand-name institutions, who are all too ordinary and in many cases have inferior problem solving skills, and in fact sap the morale of groups in which they work, due to the preferential treatment they get. I have to admit though that people with high grades tend to be careerists and 'get things done' types. Given the kind of work done by most indian software companies - mainly maintenance and enhancements, the get things done attitude find favours with managers dealing with demanding customers, mainly in western countries.

The other great desi obsession is with engineering degrees. A recent issue of Business World carried a discussion between HR heads of leading indian and MNC software houses, CEO of Mindtree and the head of the Indian School of Business , Hyderabad. The head of ISB and MNC HRs were all for diversity in the workforce - recruiting non-engineers, while the HR person from INFY played Bullshit Bingo, the chap from Mindtree typically blamed the educational system. Kiran Karnik, the head of NASSCOM surprisingly sounded rational . There seemed to a general recognition that the current policies were flawed at least amongst a section of the panel, with a MNC HR going so far as to say that recruting people who "fall through a round hole" is not good. I don't expect any change of heart on the part of indian HRs or managements. They are a reactive lot and might experience a change of heart only after losing a couple of deals to companies in Philipines, Sri Lanka or Vietnam. In any case how did this whole engineering fallacy start ? After all it is not confined to indian companies. Alistair Cockburn, the guy who started the whole agile thing , offers some fascinating insights into how software came to be regarded as an engineering subject in this interview with Doug Kaye of IT Conversations. It seems after the second world war , Physics was the most glamorous subject with very math-heavy topics quantum mechanics that started causing envy - "discipline envy" as Cockburn labels it. There were furious attempts to make every subject seem as mathematical as possible - and software development did not escape the attention of these folks who were suffering from discipline envy. The end result is a very contrived field called Computer Science/Engineering that helps many faculties with their tenure, and many airline companies make both ends meet . Paul Graham (himself a computer science Ph.D from Harvard) puts it beautifully :

I've never liked the term "computer science." The main reason I don't like it is that there's no such thing. Computer science is a grab bag of tenuously related areas thrown together by an accident of history, like Yugoslavia. At one end you have people who are really mathematicians, but call what they're doing computer science so they can get DARPA grants

Joel does point out the difference between software development and computer science/engineering.

So long as the work that was done was at the low end of the food chain, this didn't matter, but with increasing competition , increasing wage levels - the importance of creativity and problem solving skills is bound to become important in the near future - indian companies have to realize that software development skills cannot be the preserve of engineers or people with high grades. It's now possible to get software developers at 20-30% higher wages in some american states. Software developers in the US are not dumb and ultimately they will figure out a way to beat the cost advantage that Indian sw companies seemingly enjoy now. The cockiness and gloating attitude that results in a a shallow "software super power" attitude amongst many of our nouveau riche middle class folks sickens me. But hey why should I be bothered about the bigger indian companies, maybe a creative destruction of these behemoths will result in a true techie/hacker period of creativity in the Indian software market.

P.S. As you might have figured out by now, I don't have an engineering degree ;-)