I’d like to ask for suggestions and to share my experience about hiring on oDesk.
I lack time but have excess funds in my consulting business. I also have a validated idea. So I decided to try and hire a contractor on oDesk to do a prototype.
I have worked with oDesk before. I’ve hired a guy from Philippines in past, and he was reasonably good at some $20/hr, except not very pro-active in communications. I found him by doing a search on oDesk and offering him the job, and he accepted. I recently heard on some podcast (SFROU?) a better process should be used:
- Post a job
- Wait until 10-15 contractors submit their applications
- Select 3
- Give those 3 an identical task to perform (paid task, this is only fair)
- Select 1 who provided best code quality/delivery time combination
Well, sounds reasonable, and so I decided to try this process.
Posting the job
That was simple. I described the business context, gave outline of what I want to achieve and explained the process I’m going to follow to hire.
I also requested submitters to answer a few questions regarding the project, i.e. “What part of the project you expect to take most time?” and a few others.
For a large Weblogic farm, I’m looking for a tool that walks over each
domain in the list and records the current configuration values into a
timestamped snapshot (XML).
Later the operator may select two of the snapshots and obtain the
difference between the two for troubleshooting or for propagating the
changes into other domains in DEV-DIT-SIT-UAT chain.
There is no satisfactory API to collect the configuration from a
Weblogic domain, and so the collecting should be done via a web
scrapping. Good news is that all the pages in Weblogic console are built
the same way. In fact, I now have a PoC plain Java utility that does
the collection of some of the configuration entries; that utility can
be used as a reference.
The end result should be a WAR-deployable UI application that allows to
add/update/delete the Weblogic domains to track, take snapshots on
demand and show the snapshots’ diff.
I prepared the screens mockups with Bootstrap (attached).
The main skill for the developer is Java Servlets + UI development.
JSoup is nice to have, Weblogic is nice to have, basic XML is good to
The work is planned like this:
- I will collect a few of candidates and give each the same paid
pilot to do: the minimal functionality that touches all the areas that
needed for completion.
- I will evaluate the results and select one of the candidates for further development.
- The selected individual will be given tasks to continue the project
to the next milestone. The rest would get their earned money (of course)
and a honest feedback.
Almost immediately contractors started to submit.
Few notable things.
First, the range of rates was very large. The minimum was around $9/hr, and the maximum was around $30/hr. oDesk doesn’t allow to specify the target rate, only a range “low, balanced, high”. My target was in 12-20 range, so immediately some of the contractors have to be dismissed (“Hide” button on the oDesk UI).
Second, some of the contractors clearly did not took time to read the job description. Their answers to the posted questions were not fitting at all. For instance, one guy was talking about a “site” while the job is about an intranet web application.
Some other worked around the posted questions by saying “Can be discussed over Skype” for each answer. Those folks I dismissed right away. If one cannot take a minute to answer a few simple questions to get a job, how can I expect a good communication with them during the job?
Third, some of the contractors came via “agency”. I’m cautions about those. One thing, I do not want to pay extra for “overheads”. Second, I noticed that some of the agency contractors have a much higher posted rate on profile than on the submission (e.g. 30/hr vs 16/hr). That tells me it could be a bite-and-switch type of operation.
It looks like the expected way of communication on oDesk is Skype. Almost all contractors immediately give you their skype id.
Nonetheless, in about a day I had 12 candidates within my rate range and matching the basic qualifications.
Select 3 Of Them
That’s where it got complicated. How exactly I go about selecting 3 “best fitting”?
Conventional wisdom is to do a face-to-face (over Skype) interviews. I had two reasons not to do it though:
- I suck at doing interviews; did it in past and I know this is my weakness; I just cannot “read” people
- The spelling language of some of the contractors was quite bad (worse than mine) and I only expected the spoken would be worse. But as long as they can deliver the code I did not want to discount them.
- I simply did not have 12 hours to interview 12 people. Those hours I needed to bill my client.
And so I went with a more formal way. I made a spreadsheet with all the contractors where I listed their rates, ratings on oDesk, number of hours they did on oDesk, my subjective feeling if they “got” the task based on their answers, their familiarity with the technologies I needed, and so on.
I gave each of the factors a weight and calculated the overall rating. That gave me 3 best candidates. I then reviewed those candidates again to double-check my numbers against my gut feeling and they looked fine – within the right rate (not too cheap, not too expensive: all 3 was about $12/hr), with the right qualifications and with the right work history.
I hired all three for the test task, giving them the following direction:
The results will be evaluated both by code quality and the delivery time.
Code quality, in addition to obvious things like being bug-free, includes readability (i.e. self-documenting for future maintenance developers) and use of standard APIs vs DIY implementations.
“Hire” button in oDesk doesn’t close the job, so I could hire them one after another without creating a special test job for them. I set the weekly limit to 40 hours thinking this should be the maximum time a developer should spend on a basic CRUD UI, even with some extra functionality I throwed in.
I figured in the test task I want to see them touching every area that will be required in the real application. So I requested to do:
- Basic CRUD UI
- Some basic XML work
- Some use of JSoup library
- Setting up the build structure
Well, things did not go well right from the start.
One guy asked one clarification question, and then just went silent. No further questions, no progress reported. It’s only been 2 days, so I’m giving him a benefit of having a life.
Second guy was clearly confused and sent me a barrage of the questions which made me confused in turn – what exactly he doesn’t understand? After some 5-6 emails back and forth, we finally cleared his idea. He even has logged some progress. In oDesk the client can see screenshots of what the contractor is doing - the guy has spent 3 hours looking at some code examples. That’s not where I would start doing my task, but hey, everyone has their own style. Will be watching him.
Third contractor, who’s actually a young girl, asked much more pointed questions, and said she’d start working right away. Her estimate was some 40-60 hours. A bit high, I believe, but could be just a safe number for an unfamiliar technology (Weblogic). She seems skilled, I have some hopes for her.
I did not close the job when I hired those 3 guys, and some more applications were submitted. Most I hid right away (too cheap or too expensive) but one guy caught my attention. He had exactly the right qualifications – Weblogic experience, scrapping libraries – and his answers to my questions were much more reasonable than the rest. His rate was about $16, i.e. some 25% higher than other 3 guys.
I sent him a message asking what would be his estimate for the test task. He said 6-8 hours. Wut? Seriously? I have double-checked with him and decided to add him to the test as well. If he’s really that fast, it more than beats his rate increase.
To Be Continued
That was my notes so far. Should I continue to log the progress? Is it of interest to anyone?