I think it depends a little on if you’re hiring full time or part time, you didn’t really say.
For me, I’m not big on tests and tricks in general, but especially for bootstrappers hiring their first full time dev (or a long term part time). There’s so much more to your first developer than pure coding skill. They’ll probably be asked to do a lot of things like be very active in support for instance with customers that have bugs, etc.
Hiring for fit is as important as the pure technical skill.
On the technical skill front, these days generally speaking I’d want to see a fair amount of open source work. I realize not all devs contribute to open source, but the thing is if you hire one that has you don’t need any tricky tests. You can just sit there and read their actual code of things they actually worked on. That’s something no test will give you really.
You also get to see how they work with people on issues or discussions in the community, etc. For a bigger company it may not be practical to make that a criteria, but for a bootstrapper hiring your first there’s enough people out there doing it that I think it’s a good strategy.
For those who may not be aware, we’ve done it 3 times now and it’s worked out really well. We have a great dev team.