My answer will be biased (I’m the founder of the company that makes LimeLM – licensing for Windows, macOS, Linux, and BSD ).
The question you’re asking is a bit like asking “how do I build a car?”. And the answer to that is you’ll be able to build a mediocre soap box derby car in a weekend with your kid. Or if you want to build a great soap box derby card, it might take several weekends. But building a real car? A car you can drive from point A to point B in real-world conditions? It might take you years to build a very bad one from parts found. And it almost certainly would not pass an inspection (from an inspector that wasn’t a friend or who wasn’t bribed).
To state things plainly: building accurate, fast, good software licensing that doesn’t cause customers to lose their mind is hard. Very, very hard.
We’ve been selling software licensing for a bit more than a decade and our licensing is used on hundreds of millions of devices in every industry in just about every conceivable device configuration. And even today we see weird device configurations that we have to handle.
To sum up: buy something off the shelf. There are a few good licensing solutions out there. We’re one of them.
Now, digging into your questions:
I decided it is a good time to add an online activation system for my desktop tool.
Something with public/private keys to prevent a possibility of a fake activation server.
Yes, asymmetric encryption (RSA or ECC) is a start. Every programming language has some library or something built into the language that you can use. Read the documentation (don’t use Stackoverflow – there are so many bad encryption examples there it’s unbelievable).
Also, if you’re going down this rabbit hole to stop all cracking, I’ll have to break the news to you: it’s not possible to stop (or even slow down) cracking.
If you’re implementing online activation to increase your revenue from legitimate customers, then yes, you’re doing it for the correct reasons (although I’d still strongly recommend buying off-the-shelf).
But I understand that a normal user can install the program on couple of computers and change computers / update hardware from time to time.
That’s the whole ballgame. Correctly “fingerprinting” computers (and handling legitimate changes to the same computer over time) is what separates garbage licensing products from licensing that is actually good. Honestly, you need to take all of the components of the computer, and correctly “weight” them, and generate a unique ID from those components (while still accounting for legitimate changes).
My advice: buy 20 or 30 computers and install a variety of OS systems that you’ll be supporting. Then buy a handful of external network adapters, harddrives, and other components to plug / unplug to/from these devices. That will get you a very bare-minimum start of the many weird configurations that your customers will have. Now you have to write the first-draft algorithm.
How many activations per week/month/year/total should an online activation system allow?
For a well written licensing product? 1 activation per machine. And you can optionally allow customers to move between machines.
IMO the online activation system should also track that some PC are not in use anymore and somehow should “forgot” about these old activations.
With floating licensing that’s possible. With node-locked licensing what you’re describing is impossible. I can dig into this and explain it in-depth if you’d like. You can read our article about the difference between floating licensing and node-locked licensing and you can probably figure out why it’s impossible.