.NET Forum / Languages / C# / March 2008
Identical binaries from same source code
|
|
Thread rating:  |
tararot2 - 12 Mar 2008 17:27 GMT Hi, I have to homologate my binaries so I need them to be byte-per-byte identical after each compilation (if the source code is the same, logically). Does anybody know how to achieve that with either compiler options or modifying some parts of the binaries? After some research I've succesfully patched: - timestamp: in 0x88 address, I have set to zero. - GUID after <PrivateImplementationDetails>: find this text in file and set GUID to zero. but there is an unknown data block after what I supose to be the strings declarations and I don't know what to do with. In some cases, setting it to 0's works but in other application hangs. I cannot find a generic way to clean it. I need to do that automatically for generic binaries of .NET.
Thanks in advance.
Cowboy (Gregory A. Beamer) - 12 Mar 2008 18:29 GMT I do not know everything that might be slightly different. I know that culture changes can affect things, as can the build numbers.
I am not sure why you need this, however, as you can sign an assembly to show it has not been tampered with. That may not be your reason, however, so I may be being a bit presumptious. :-)
 Signature Gregory A. Beamer MVP, MCP: +I, SE, SD, DBA
*************************************************
| Think outside the box! *************************************************
> Hi, > I have to homologate my binaries so I need them to be byte-per-byte [quoted text clipped - 12 lines] > > Thanks in advance. Peter Bromberg [C# MVP] - 12 Mar 2008 21:06 GMT Why do you "have to" homologate your binaries? And why must they be byte-for-byte identical? Is there not some other way, such as what Gregory suggested, to verify your assemblies? -- Peter Site: http://www.eggheadcafe.com UnBlog: http://petesbloggerama.blogspot.com Short Urls & more: http://ittyurl.net
> Hi, > I have to homologate my binaries so I need them to be byte-per-byte [quoted text clipped - 12 lines] > > Thanks in advance. Lasse Vågsæther Karlsen - 12 Mar 2008 23:14 GMT > Hi, > I have to homologate my binaries so I need them to be byte-per-byte [quoted text clipped - 12 lines] > > Thanks in advance. If the source code doesn't change, and you want the same binary as you originally built, surely it would be easier to just keep a copy of the original binary so that you don't have to rebuild it?
If not, why this odd requirement? What problem are you hoping to solve with it?
 Signature Lasse Vågsæther Karlsen mailto:lasse@vkarlsen.no http://presentationmode.blogspot.com/ PGP KeyID: 0xBCDEA2E3
snamds@gmail.com - 13 Mar 2008 09:12 GMT Hi, I belong to the same development group that tararot2.
Our software must be homolagated by an external company because of some legislation of our business sector. The aplicattions we make go to an specific hardware. The homologation company compiles our code, build the binaries and homolagates a "crc" of the result. At this point we could use this binaries that they compiled to be distributed to this specific hardware but the problems comes in the future. If a customer report a minor error we have to be able to fix it, build the binaries, send the result to our customer and after that send the code to homologate again. The binaries the homolagation builds must be the same that what we send to our customer.
Now we're trying to achive our goal using mono-cecil.
Thanks in advance
Lasse Vågsæther Karlsen - 13 Mar 2008 12:35 GMT > Hi, I belong to the same development group that tararot2. > [quoted text clipped - 13 lines] > > Thanks in advance But the question still stands, after you, or that external company, has built the code, isn't this binary what you would want to send to a customer?
What I'm saying is that it might be easier for you to just set up routines to retrieve the binary they build and release this as your own, instead of trying to circumvent how the linker works.
 Signature Lasse Vågsæther Karlsen mailto:lasse@vkarlsen.no http://presentationmode.blogspot.com/ PGP KeyID: 0xBCDEA2E3
snamds@gmail.com - 13 Mar 2008 13:11 GMT You are right but if there is a bug and I can generate always the sames binaries, I can fix it and send the binary to my customer in only one day. If I fix the error, send it to homologate and wait until the homologation company builds the binary, my customer don't get a fast support from me. Homologation can be very slow.
It's "just" a question of time. Very important for us and our customers.
Jon Skeet [C# MVP] - 13 Mar 2008 13:28 GMT > You are right but if there is a bug and I can generate always the > sames binaries, I can fix it and send the binary to my customer in [quoted text clipped - 4 lines] > It's "just" a question of time. Very important for us and our > customers. It's still not at all obvious to me why you can't send the original file. Why rebuild something in order to produce something that is guaranteed to be the same as another copy you've already got?
I've a feeling there's something in this situation which you haven't explained, otherwise it doesn't make sense.
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk
snamds@gmail.com - 13 Mar 2008 15:21 GMT I've nothing to hide. If there is a bug in the code i want to build a fixed version and install it in the customer ASAP. After that i'll send my source code to homologate. They'll validate the source and once validated they'll build it. So if his generated exe is not identical to the one i sent to my customer, my customer will not have an homologated version of the software.
When I say an exe is homologated I mean that the "CRC" of the exe is homologated. A customer CANNOT have an exe whose "CRC" is not homologated. That's why my generated exe must be identical byte-for- byte with their generated exe.
> <sna...@gmail.com> wrote: > > You are right but if there is a bug and I can generate always the [quoted text clipped - 16 lines] > Jon Skeet - <sk...@pobox.com>http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet > World class .NET training in the UK:http://iterativetraining.co.uk Jon Skeet [C# MVP] - 13 Mar 2008 15:34 GMT > I've nothing to hide. Sorry, I didn't mean to imply that you did - merely that we hadn't got enough of the information to understand the problem.
> If there is a bug in the code i want to build a fixed version and > install it in the customer ASAP. > After that i'll send my source code to homologate. They'll validate > the source and once validated they'll build it. So if his generated > exe is not identical to the one i sent to my customer, my customer > will not have an homologated version of the software. Right, I see.
> When I say an exe is homologated I mean that the "CRC" of the exe is > homologated. A customer CANNOT have an exe whose "CRC" is not > homologated. That's why my generated exe must be identical byte-for- > byte with their generated exe. Right. In that case, I don't think the question as originally asked would actually help you. It doesn't help for you to be able to rebuild the same source to the same binaries multiple times if the homologating company then uses a different mechanism to compile your code.
Now, three questions arise:
1) If you send the same source to the homologating company twice, do *they* end up with the same binaries?
2) Are they happy to share their homologation methods with you?
3) If you can produce the same binary as the homologation company, doesn't that defeat the purpose of the legislation? I would expect the point to be similar to crypto-signing - e.g. to guarantee that the homologation company has the original source code.
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk
snamds@gmail.com - 13 Mar 2008 16:21 GMT > Right. In that case, I don't think the question as originally asked > would actually help you. It doesn't help for you to be able to rebuild > the same source to the same binaries multiple times if the homologating > company then uses a different mechanism to compile your code. This is not a problem. We provide the compiler and method of compilation to the homologation company. They first validate the method of compilation.
> Now, three questions arise: > > 1) If you send the same source to the homologating company twice, do > *they* end up with the same binaries? > 2) Are they happy to share their homologation methods with you? 1 and 2 -> Yes, because of my prior explanation.
> 3) If you can produce the same binary as the homologation company, > doesn't that defeat the purpose of the legislation? I would expect the > point to be similar to crypto-signing - e.g. to guarantee that the > homologation company has the original source code. No, because we must explain what we have post changed in the binary and they validate the method. The don't have to believe in what we say, they can check the method.
I'm sure this is the way we must do things. The problem is that this is our first project with C#. Before, with C++ we only had to clear the timestamp of the binary.
Thanks for your time.
Jon Skeet [C# MVP] - 13 Mar 2008 16:36 GMT > > Right. In that case, I don't think the question as originally asked > > would actually help you. It doesn't help for you to be able to rebuild [quoted text clipped - 4 lines] > compilation to the homologation company. They first validate the > method of compilation. In that case, would it be possible to have a cache of source code + binaries, so that it always consults the cache before bothering to build at all?
The best way of making sure that you get the same result is not to rebuild unnecessarily :)
I do understand that that may not be feasible, however.
> > Now, three questions arise: > > [quoted text clipped - 16 lines] > is our first project with C#. Before, with C++ we only had to clear > the timestamp of the binary. The CLI spec gives details of the PE format, but it's possible that there are some implementation-specific fields that are being generated every time.
By the way, I wouldn't set the GUIDs and times to zero - I'd choose appropriate values and always reuse those.
Note that if you sign your assemblies, all of this editing will probably (hopefully, even) invalidate the signature - in which case the suggestion at the top of this post is probably the only practical one.
Would it be possible to make the CRC comparison ignore specific parts of the binaries, so long as you could explain that they don't affect the behaviour of the code? That way we wouldn't need to work out what values to use for the new binaries, just reasons why they're not important.
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk
snamds@gmail.com - 13 Mar 2008 17:42 GMT > The CLI spec gives details of the PE format, but it's possible that > there are some implementation-specific fields that are being generated > every time. We want this to be our last alternative.
> By the way, I wouldn't set the GUIDs and times to zero - I'd choose > appropriate values and always reuse those. You are right. We are setting zeroes in our tests. The idea is to reuse good values as you say.
> Note that if you sign your assemblies, all of this editing will > probably (hopefully, even) invalidate the signature - in which case the > suggestion at the top of this post is probably the only practical one. We don't need to sign the assemblies.
> Would it be possible to make the CRC comparison ignore specific parts > of the binaries, so long as you could explain that they don't affect > the behaviour of the code? That way we wouldn't need to work out what > values to use for the new binaries, just reasons why they're not > important. This is not a solution, they must CRC it easily. We can't control this process.
We are getting good results using mono-cecil. It's almost done.
Thank you all very much.
Jon Skeet [C# MVP] - 13 Mar 2008 17:47 GMT > > The CLI spec gives details of the PE format, but it's possible that > > there are some implementation-specific fields that are being generated > > every time. > > We want this to be our last alternative. Consulting the spec as the last alternative? To be honest, it'll be a lot more authoritative than other answers here.
Another alternative might be to look at the Mono compiler source code - that will hopefully make it clear what belongs where.
> > By the way, I wouldn't set the GUIDs and times to zero - I'd choose > > appropriate values and always reuse those. > > You are right. We are setting zeroes in our tests. The idea is to > reuse good values as you say. Goodo.
> > Note that if you sign your assemblies, all of this editing will > > probably (hopefully, even) invalidate the signature - in which case the > > suggestion at the top of this post is probably the only practical one. > > We don't need to sign the assemblies. That definitely makes it simpler.
> > Would it be possible to make the CRC comparison ignore specific parts > > of the binaries, so long as you could explain that they don't affect [quoted text clipped - 6 lines] > > We are getting good results using mono-cecil. It's almost done. Okay, glad to hear it's going well.
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk
snamds@gmail.com - 13 Mar 2008 18:20 GMT > Consulting the spec as the last alternative? To be honest, it'll be a > lot more authoritative than other answers here. I meant we prefer to use well tested third libraries rather than manually parse the binaries by ourselves.
> Another alternative might be to look at the Mono compiler source code - > that will hopefully make it clear what belongs where. We have made changes to the mono-cecil source code to achieve our goal.
> > > Note that if you sign your assemblies, all of this editing will > > > probably (hopefully, even) invalidate the signature - in which case the [quoted text clipped - 3 lines] > > That definitely makes it simpler. I don't get the point. How can signing the assemblies would help us? We can't change the homologation rules. It must be done by CRC as i've explained.
Jon Skeet [C# MVP] - 13 Mar 2008 18:28 GMT > > Consulting the spec as the last alternative? To be honest, it'll be a > > lot more authoritative than other answers here. > > I meant we prefer to use well tested third libraries rather than > manually parse the binaries by ourselves. Right - yes, that's a laudable tactic.
> > Another alternative might be to look at the Mono compiler source code - > > that will hopefully make it clear what belongs where. > > We have made changes to the mono-cecil source code to achieve our > goal. Right.
> > > > Note that if you sign your assemblies, all of this editing will > > > > probably (hopefully, even) invalidate the signature - in which case the [quoted text clipped - 7 lines] > We can't change the homologation rules. It must be done by CRC as i've > explained. No, the fact that the assemblies *aren't* signed makes it simpler.
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk
snamds@gmail.com - 13 Mar 2008 18:53 GMT Ok, all clear now.
Thank you very much Jon.
Ben Voigt [C++ MVP] - 13 Mar 2008 21:42 GMT > I don't get the point. How can signing the assemblies would help us? > We can't change the homologation rules. It must be done by CRC as i've > explained. CRC is not a good hash. If you predict the file the homologation procedure creates you can construct one with arbitrary behavior and the same CRC.
snamds@gmail.com - 14 Mar 2008 09:35 GMT > > I don't get the point. How can signing the assemblies would help us? > > We can't change the homologation rules. It must be done by CRC as i've > > explained. > > CRC is not a good hash. If you predict the file the homologation procedure > creates you can construct one with arbitrary behavior and the same CRC. Sure it's not a CRC. I have a bad speaking habit. I'm sorry if it's have been confusing
Ben Voigt [C++ MVP] - 13 Mar 2008 15:36 GMT >> Hi, I belong to the same development group that tararot2. >> [quoted text clipped - 17 lines] > has built the code, isn't this binary what you would want to send to > a customer? They have to prove that the binary installed on the machine matches the source code.
Legislation of business sector probably means this is an electronic voting machine, although there could be similar requirements in other markets.
snamds@gmail.com - 13 Mar 2008 16:29 GMT > Legislation of business sector probably means this is an electronic voting > machine, although there could be similar requirements in other markets. It's not for a voting system. It's a multiplayer casino-like game machine.
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|