Preventing manufacturing service from copying hex code

[mikeselectricstuff]

A lot of big production runs at EMS places use pre-programmed MCUs, and its a huge pain. Not only is it messy unpacking parts, programming them in jigs, and then repacking them suitable for the PnP machines on the EMS line, you also get a higher defect rate if the device has legs. There will always be some rate of leg bending with all that extra handling. QFNs might be a boon here.

microchip programs the parts for you, they aren't going to screw anything up..

https://www.microchipdirect.com/programming/

Various vendors will do the programming for you at the factory, and mark the chips with a customer specific code. They don't all mention this. You have to talk to the sales people, and you may need to be buying in considerable volume. However, this usually requires considerable setup time, and most people are trying to churn out some boards the moment they are happy with the design. The time issues means programming before the chip leaves the factory is not for everyone.

In the case of Microchip direct, I think the only up-front cost now $29 setup- ISTR they used to have a minimum order of either $150 or 150 parts, don't recall which, but can't see this mentioned any more

[coppice]

A lot of big production runs at EMS places use pre-programmed MCUs, and its a huge pain. Not only is it messy unpacking parts, programming them in jigs, and then repacking them suitable for the PnP machines on the EMS line, you also get a higher defect rate if the device has legs. There will always be some rate of leg bending with all that extra handling. QFNs might be a boon here.

microchip programs the parts for you, they aren't going to screw anything up..

https://www.microchipdirect.com/programming/

Various vendors will do the programming for you at the factory, and mark the chips with a customer specific code. They don't all mention this. You have to talk to the sales people, and you may need to be buying in considerable volume. However, this usually requires considerable setup time, and most people are trying to churn out some boards the moment they are happy with the design. The time issues means programming before the chip leaves the factory is not for everyone.

In the case of Microchip direct, I think the only up-front cost now $29 setup- ISTR they used to have a minimum order of either $150 or 150 parts, don't recall which, but can't see this mentioned any more

I was only thinking of the setup time, not the cost. What typical lead time do you see for pre-programmed devices from Microchip?

[OwO]

In the part of the world I plan to sell this product, it is too difficult to get a favourable judgement from court.

If you are trying to sell in China (or similar places) my advice would be to know the market well and price your product competitively. There is no bigger target for "replacements" or "clone" development than a foreign product with bloated sky-high margins (assuming there is demand for your product that is). See: FTDI. If multiple Chinese firms can develop from scratch stm32 compatibles then I don't see how they can't reimplement some simple PIC firmware. There used to be a TI INA199 Chinese clone, but that's being discontinued. Why? TI is now selling genuine INA199s at a lower price.

[radiolistener]

You can use hardware lock to processor serial number. So, the manufacturer will send you serial number of processor and get firmware locked for specific instance of processor. But this solutions won't works with usual controllers such as PIC or STM32, just because code can be easily disassembled, analyzed and patched.

Such solution can works with FPGA gateware, because it will be hard to disassemble and analyze gateware.

[mikeselectricstuff]

A lot of big production runs at EMS places use pre-programmed MCUs, and its a huge pain. Not only is it messy unpacking parts, programming them in jigs, and then repacking them suitable for the PnP machines on the EMS line, you also get a higher defect rate if the device has legs. There will always be some rate of leg bending with all that extra handling. QFNs might be a boon here.

microchip programs the parts for you, they aren't going to screw anything up..

https://www.microchipdirect.com/programming/

Various vendors will do the programming for you at the factory, and mark the chips with a customer specific code. They don't all mention this. You have to talk to the sales people, and you may need to be buying in considerable volume. However, this usually requires considerable setup time, and most people are trying to churn out some boards the moment they are happy with the design. The time issues means programming before the chip leaves the factory is not for everyone.

In the case of Microchip direct, I think the only up-front cost now $29 setup- ISTR they used to have a minimum order of either $150 or 150 parts, don't recall which, but can't see this mentioned any more

I was only thinking of the setup time, not the cost. What typical lead time do you see for pre-programmed devices from Microchip?

In my experience of a handful of orders, if you bypass the sample verification stage, it can be under a week from starting the process to delivery of few hundred parts. Up to a couple of weeks for a few thousand. Opting for verification samples adds maybe a week .

[Marco]

Just curious, what does it cost to get the firmware extracted for your particular device?

[ataradov]

This is a service that has been around for ages, so they may actually be legit https://russiansemiresearch.com/ They say that it costs $500 to extract almost any PIC. Some go to $1000.

[luiHS]

 
Encrypted bootloader + Firmware with blacklist (to detect pirate boards by serial number). The blacklist is only possible with microcontrollers that have a unique factory serial number, virtually all ARMs.

You can also use the NXP RT series with encrypted images, but that implies that the microcontrollers must have previously been initialized with the unique key, before attempting to record the firmware.

In any case, by brute force, dissolving the encapsulation with acid, and deactivating the read protection bit, virtually any microcontroller can be read. Some manufacturers advertise protections against this type of illegal access, but the truth is that today almost all microcontrollers are broken by the Chinese and they give you the object code, for a few hundred dollars in cost.

[KL27x]

What happens mid production cycle?
Don't all you guys have a CNC chip flashing robot? You upload the firmware and stick your chips in one end, then you press a button and all your chips are flashed and come out the other end?
That can only cost 10 square feet and $60K, right?

What if i told you there was a fairly simple way to re-flash a ton of IC's with fairly minimal cost... I've had it for over a decade, and I've flashed probably over 100K chips with this method. But I still haven't found a way to make money from it, so it's just an inhouse tool for me. I'm sure lots of businesses use their own inhouse solutions for this kind of problem.

Yeah, bootloader is fine. But it still costs. I think the minimum that Microchip will do it is around 10 cents per device, even if you ignore the setup fee per run. It costs 3-4 cents just for them to put a dot of paint on the part. That would be extra. Times 100K is like 10 grand, so far, I can say I saved. Plus having the flexibility and control of not giving out firmware.

[I wanted a rude username]

I did consult a lawyer regarding IP protection and stuff. Their advice is to do something technical to avoid the software getting leaked at the first place. In the part of the world I plan to sell this product, it is too difficult to get a favourable judgement from court.

Yeah, trademarks can be a bigger threat. In China, it's a first-to-file system ... doesn't matter if you have a trademark on the product in Canada, if someone files in China before you they get the trademark, they can start selling their copies, and they can even sue to stop your production of devices that now infringe on their trademark.

Like many have said, the firmware can be redeveloped. You have to know the threat model.

[james_s]

What happens mid production cycle?
Don't all you guys have a CNC chip flashing robot? You upload the firmware and stick your chips in one end, then you press a button and all your chips are flashed and come out the other end?
That can only cost 10 square feet and $60K, right?

What if i told you there was a fairly simple way to re-flash a ton of IC's with fairly minimal cost... I've had it for over a decade, and I've flashed probably over 100K chips with this method. But I still haven't found a way to make money from it, so it's just an inhouse tool for me. I'm sure lots of businesses use their own inhouse solutions for this kind of problem.

Yeah, bootloader is fine. But it still costs. I think the minimum that Microchip will do it is around 10 cents per device, even if you ignore the setup fee per run. It costs 3-4 cents just for them to put a dot of paint on the part. That would be extra. Times 100K is like 10 grand, so far, I can say I saved. Plus having the flexibility and control of not giving out firmware.

A local manufacture I did some contract work for while I was between jobs would bring his kids in and set them to work doing this. I set up the programming jig and then one kid would load a PCB in the jig and hold it there while the other hit the program button on the laptop. It actually worked better than I'd have expected.

[mikeselectricstuff]

I think the minimum that Microchip will do it is around 10 cents per device, even if you ignore the setup fee per run. It costs 3-4 cents just for them to put a dot of paint on the part. That would be extra.

ISTR it's lower than that for the smallest parts, but if you're doing 100K units you can afford to spend a few grand on a streamlined production-line programming system, though even then it likely still adds some time, which may or may not be an issue.
For a few thousand units, a few hundred to have your parts come in programmed and marked is very good value.

[KL27x]

^ That might be a good value. Depends on how much time it takes to flash them. Esp if you are only putting a short bootloader on there, you might be paying $450.00 for something that could take minutes of very low stress/attention labor. If you use the right equipment, flashing a chip could literally take the amount of time to flash plus a quarter of a second. For even higher volumes, automation is not difficult to add. Maybe your time is more valuable. But if I can save >$200 an hour while watching Youtube, AND gain flexibility to mid-production changes, I think it is worth it. This also saves money when sourcing of the IC might be an issue, and you might have to change to different variants of the IC. Microchip will give you a new setup fee and pricing for the smaller amounts you are temporarily using.

A local manufacture I did some contract work for while I was between jobs would bring his kids in and set them to work doing this. I set up the programming jig and then one kid would load a PCB in the jig and hold it there while the other hit the program button on the laptop. It actually worked better than I'd have expected.

And no, that's not what I'm talking about. But I guess if you have free labor, why not use it? Sure, sometime you have to deal with individual assembled PCBs, for JTAG testing or for repair or update, etc. But PCB's aren't all a uniform size and shape, so this often going to cost in manual labor.

The alarm bells began to ring when the OP said 'buyer/reseller insisted on having it made at their preferred EMS' and were deafening when 'has to be repaired there too'.

I agree, it is good to be a little paranoid, here. I have BT, too. Customer telling me the manufacturer needs the firmware, today, for testing the latest prototype. No discussion prior. But today, the protos have arrived from China, and the manufacturer in America needs it ASAP.

Me: uh, no. You do not give your firmware to a fly by nite very small design and manufacture company under this kind of duress. Total BS. I wrote a firmware for testing-only, and gave him this. Later, public records show that the owner of this very young company has 3 prior companies all under litigation for exactly this kind of IP theft. We were both on site, at one point and he asked many times for firmware and even to regale to me how easy it is to lift the firmware off a chip, anyways. He enticed me of tales of a company he knows (unnamed) that will flash chips cheaper than Microchip. And he continued to ask for the firmware. He also asked me to modify some disassembled code because one of his other clients was having problems with unlicensed cloners, so his client wanted to make their code different. By paying him to pay me to alter a disassembly. LOL.

I think that in many cases where firmware is lifted from a chip, the entire image is not lifted. There will be probably some configuration bits that are not readable. And possibly it takes overlap of many images to get a near complete picture. IOW, I think even if you pay for some service to get a firmware image, you might in many cases still have some work to do to fill in some blanks. And hopefully by then, the firmware is outdated and worthless, or less at least less valuable.

Also for legal reasons, I think it is important to not give firmware to another company without some legal documentation of the purpose and limitations. There is a saying here. Possession is 90% of ownership. Despite what was said, the court will be happy to suck away your money to clarify what was the original agreement when you intentionally gave the other entity your firmware. They know you can sue if you catch them. But by the time that goes anywhere, this company can be long gone and they can make a new one. It's like if you let someone stay at your house for "a few days," that is legally where they live, and you have to go through an eviction if they decide to stay longer. The cops literally ask questions like "did you give him a key? Does he have any belongings here, even a toothbrush? Ok, then, he's your problem until you go through the legal process of evicting him. You can't deny him entry."

[NorthGuy]

For a few thousand units, a few hundred to have your parts come in programmed and marked is very good value.

But you still give your firmware to Microchip, where dishonest employees or hackers can steal it. This is certainly not very likely, but still, this is not 100% protected neither.

[KL27x]

Microchip is a very large company that depends on its reputation and history. It also does not do hardware design/manufacture/testing of the products these IC's go into. Unlike the manufacturing/testing/repair subcontractor who has all the other keys to cloning your product. Right now. And whenever your company spends money on advertising, they can potentially cash in on the wave and the worry about the wrist slap and court appearances, later, while you spend money to prove that the way he is using your firmware and hardware was not part of agreed compensation.

Do you think the court and lawyers care about justice? They see cash. They don't care which side is right, just how much money they will get out of the mess.

[Howardlong]

For a few thousand units, a few hundred to have your parts come in programmed and marked is very good value.

But you still give your firmware to Microchip, where dishonest employees or hackers can steal it. This is certainly not very likely, but still, this is not 100% protected neither.

I’ve used Microchip Direct for about a decade for mass programming my products, many tens of thousands of devices. They are dependable and get the job done.

I will also reiterate and confirm all of Mikeselectricstuff’s comments, including bypassing the verification stage, and having them direct ship to your assembler.

Regarding the verification bypass, I have had situations where Is a part shortage, but the next memory version up from the same family was available, so I issued the same firmware rebuilt for the larger chip, tested locally, and committed directly to Microchip Direct without verification, works a treat. Same applies if only extended temperature versions are available. Sure it might cost a few pennies, but it’s either that or it’s no product.

Similarly, I have built my own jigs to manually bulk program chips, in extreme shortage situations. It’s not a career I can recommend.

[KL27x]

Regarding the verification bypass, I have had situations where Is a part shortage, but the next memory version up from the same family was available, so I issued the same firmware rebuilt for the larger chip, tested locally, and committed directly to Microchip Direct without verification, works a treat. Same applies if only extended temperature versions are available. Sure it might cost a few pennies, but it’s either that or it’s no product.

But you have to pay the setup fee again, and pay for the per unit price at what may be a much smaller quantity. Also, Microchip may be out of an IC for the next 12 weeks while there is a supply at Digikey.

Similarly, I have built my own jigs to manually bulk program chips, in extreme shortage situations. It’s not a career I can recommend.

Depends how good you are at building jigs. I think I could sell my designs. I'm not that keen on the business aspect, though. Patents just don't seem to be useful to non-mega corporations.

I'm pretty sure whatever "jigs" Microchip uses, they are not doing this at a net loss. They are managing to make it profitable. I don't think they are putting their kids to work manipulating individual chips for pennies. I imagine they use a bit of a better jig than you have thus far made.

Not a career? Well, poo poo all you want. Even if I make $500.00 an hour to do some other aspect of my work, I will still take $200/hr to watch Youtube. Or give that job to someone I know, pay him less than $200/hr, and keep the difference. I could be pocketing this kind of money right now, while dicking around on the forum.

[NorthGuy]

I'm pretty sure whatever "jigs" Microchip uses, they are not doing this at a net loss. They are managing to make it profitable. I don't think they are putting their kids to work manipulating individual chips for pennies. I imagine they use a bit of a better jig than you have thus far made.

I don't think the programming comes at extra cost to them. They still need to power chip at some point, test it, program system flash etc. Adding the user program to the mix costs nothing to them.

If they were doing it with stand-alone programming machines, it would cost more.

[KL27x]

And you think they use children to do that part, too, picking up chips and putting them in a jig? Or do you think they have automated and simplified it with purposeful engineering?

  If they were doing it with stand-alone programming machines, it would cost more.

I would bet a house that they program the chips AFTER they have tested and stamped them with the QA seal of approval. I do not for a second believe they inject small batches of code from thousands of customers into their production line. The production line pumps out blank chips. They flash them from stock. Bet you a house.

When the production line is up, it is to make one thing. They don't enter some code for 10,000 of them and package those up separately. When the production line is up, it is to spray out blank chips like out of a firehose. Custom programming is done separately, after the fact. They don't slow down the entire production line at a bottleneck because it takes 25 seconds per chip to flash some customer's firmware, the entire lot of which is a small fraction of a day's production. It is cheaper for them to take the blank ammo and then flash it, after the fact, in a separate and flexible and paralellable step utilizing multiple machines. I bet.

Doing it during production just makes too many problems, just even with lead time. Well, yeah, you submitted your hex and paid the fee, but we.... aren't even making another run of these chip for another 3 months, so just wait.

An IC manufacturer sets up the factory for one chip. Runs that for X time. The changes over to another product line, pumps those out. It's not sitting there waiting for orders to come in. Anything custom will happen after the fact, maybe months or years after the chip was originally produced and verified. There is not 9 billion semiconductor plants, one for each semiconductor in Digikey's catalogue.

It is much more likely, IMO, they have several reprogrammable/configurable machines, at least a few for each package type, that they can use for these custom firmware orders if and when the come in.

[NorthGuy]

When the production line is up, it is to make one thing. They don't enter some code for 10,000 of them and package those up separately. When the production line is up, it is to spray out blank chips like out of a firehose.

Why do you think it's done all at once. For example, there are device families where all the PICs from the same family are likely to have the same die, but then they're configured as different parts. They can produce bunch of pro-PICs and then do the configuration and engraving at a later date, depending on the demand.

[KL27x]

^Do you think Microchip has a single dedicated production line for every one of their products? That when you put in a custom order for specific PIC in specific package, it happens to be in this "verification, powerup, test it" stage, in parallel with every other single programmable part in Microchip' entire inventory?

Do you think that equipment just sits around covered in dust until an order comes in, then someone walks over to the one out of thousands of PIC micro production lines and fires it up to complete your order?

Or do you think they maintain a variety of unique products on a smaller number of machines? And they rotate through them, as needed? I bet you can custom order firmware for a PIC that hasn't been produced in decades. Do you think Microchip keeps a stockpile of raw untested stock, just in case someone custom orders firmware? Or do you think they might keep a single inventory of blank chips, ready to sell in either form? Once the candle is lit, you get as much done as possible. And custom orders aren't conducive to that.

IMO, it seems more likely they have some number of flashing machines for each package type (to cover scores or hundreds of their products in this particular package size) which can be custom loaded and programmed per order (hopefully integrated into the online info you have to fill out and the hex you submit), to the tune of the $150.00 setup cost for the guy that has to load the machine with that particular chip and to push the right buttons in the right order. Of course that includes the machinery to package into tape and reel, if need be, which my machines do not do tape and reel.

But this is my opinion. I don't know how Microchip operates.

[NorthGuy]

Or do you think ...

No. I don't think any of these things. I don't know how their process is organized. And neither do you.

However, since they manufacture PICs, they are in position to find a better way to pre-program chips than a third-party vendor, such as DigiKey. I also think they have found it. Hence the excellent low cost programming service.

[KL27x]

Well, if you think Microchip only manages to charge "so little" (say around 10 cents per, depending on device and quantity) because they do it while making/testing the chip in the first place?

I don't know how Microchip operates, but this part is wrong. It doesn't cost this much to flash microcontrollers. And I bet you can find companies to do it cheaper, even if you don't buy the chips from them. We have the technology that we don't need to use our kids to do this. This is definitely an add-on that is profitable to Microchip, even if they do it after the fact with finished-product, consumer-ready chips.

But Microchip can charge what they do because of trust and convenience.

[Bud]

I'm pretty sure whatever "jigs" Microchip uses, they are not doing this at a net loss. They are managing to make it profitable. I don't think they are putting their kids to work manipulating individual chips for pennies. I imagine they use a bit of a better jig than you have thus far made.

I don't think the programming comes at extra cost to them. They still need to power chip at some point, test it, program system flash etc. Adding the user program to the mix costs nothing to them.

If they were doing it with stand-alone programming machines, it would cost more.

I've seen how chip cards are made and it is similar process, the machine assembles a product from components, i.e. the chip itself, the card carrier, magnetic stripe where needed, hologram stickers, etc. When a batch is running you do not want to slow it down or God forbid to pause it   for whatever reason. There are schedules for production with timing for each batch job. After being made the finished products are put in the vault. For custom programming the specified number of cards is taken from the vault and gets  programmed on a separate much smaller scale and lower cost equipment. 

[KL27x]

Yes. But NorthGuy's point is that these cards can be tested and programmed at the same time. Apparently before even capping them, since we're talking about "cards?"

I agree this is technically possible, but it is highly impractical just due to the sheer number of unique products that a company like Microchip sells and supports. I think it will be cheapest for them to batch out ALL stages of the process to the finished product. And then programming is separate. But then I am supposing that this programming is a small part of their volume.

Perhaps a company like Renasas does the programming in conjunction with some other stage, but Renasas might at any given time have 3 customers taking up 99% of their production across only a few scores of different products and hexes. It's not like making cars that are each made to order; making teensy eensy chips that are easy to transport and handle (by man and machine), it is probably cheaper to finish the jobs that have to eventually be finished then put them in a box until they're called upon.

Just conjecture and opinion, but I agree this is likely:

gets  programmed on a separate much smaller scale and lower cost equipment.

The testing equipment for the various micros is likely very much more involved and taking a lot more time to switch/setup between different products; there has to be calibration and verification of settings, QA samples for destruction testing, blah blah. Any one mistake that slips by and goes for an hour might cost millions of dollars. So they will do just the one product for a period of time before switching to another. Can't just do any one on demand. OTOH, flashing is very easy. Package size is standardized. Just load the right chip, configure the software correctly, then push the buttons. Can be switched between different chips and hexes very quickly, and output speed not that critical*, cuz these devices will be relatively inexpensive and require a very low amount of human oversight to operate. One person can keep many of them fed and churning on his watch, keeping the hoppers fed and clearing the occasional jam.

*As I suggested before, it could be essentially limited to the time it takes to flash a chip or set of chips plus w/e for the maintenance, operation, and setup.