.NET Forum / .NET Framework / Distributed Applications / December 2003
Non-trivial development using Visual Studio 2003 .NET
|
|
Thread rating:  |
michaelj - 25 Nov 2003 21:11 GMT My questions are directed to those who are successfully doing non-trivial "enterprise" development using Visual Studio 2003 .NET. It has been my recent experience that this development environment works fine, as long as all the development uses managed components. However as soon as one needs to leverage unmanaged code, some of the processes it implements start to break down.
My question is this: How does one structure multiple solutions (with multiple projects each), to best facilitate development and testing?
I read, with enthusiasm, the white paper titled "Team Development with Visual Studio .NET and Visual SourceSafe". Many of my questions about the structure of solutions in Visual Studio .NET were answered. I recommend this document. I am attempting to scale this solution up to a large enterprise application but so far I have not had much success.
In the past, using Visual Studio 6, we would jumpstart our development by starting with an install of the previous nights build. Then new development would proceed with each developer building the components they were interested in. We have every project setup so that its output location coincides with the location defined by the installer. Dependencies that are not a part of the current workspace are satisfied by the components installed by the installer. This process has worked great. Great until Visual Studio .NET that is.
It seems that Visual Studio .NET does not like it if you fiddle with the Output Path property of managed projects and build all outputs to the same location. This precludes my old process from working. At first blush that seemed fine as they invented another mechanism to copy managed dependencies based on information in the project files. With this mechanism managed dependencies magically show up every place they need to be.
Unmanaged dependencies are a different story however. It seems that they work much as they did before meaning that I can change the Output Path property to put the output wherever I need. However, between the final executable and various unit tests (which are all implemented in managed code) I find myself with eight or ten copies of the managed dependent components. All which need their own copies of the unmanaged dependencies if they are to be tested.
We have been discussing this problem amongst several of our most seasoned developers looking for ideas how we might implement a new process. We have tried changing the Output Path property and we found out why we were told not to. We invented a complex set of batch files that we execute with post build steps. This works, but it is cumbersome and we are not convinced that it represents a long term solution. We are hoping that others might have discovered that which we have failed to.
I find it hard to accept, but I don't know how Microsoft could expect development of non-trivial applications to succeed, given what I know of how this development environment works.
To repeat my question, how does one structure multiple solutions (with multiple projects each), to best facilitate development and testing?
If you've gotten this far you have my sincere thanks. Please post replies to the news group so that others may benefit.
Michael
Donald Kackman - 25 Nov 2003 22:02 GMT You know, I am stuggling with this very issue. I have also found that VS.NET really starts to break down as a conifuration management tool for complex n-tier applications.
I just posted a similar topic at codeproject.com. (http://www.codeproject.com/script/comments/forums.asp?forumid=3831&select=6 75694)
Perhaps an answer to both of our questions will come from this group or codeproject!
cheers, don
> My questions are directed to those who are successfully doing non-trivial > "enterprise" development using Visual Studio 2003 .NET. It has been my [quoted text clipped - 55 lines] > > Michael Jeff K - 02 Dec 2003 23:16 GMT HI,
My team of 6 developers has been working on a multi- solution, multi-project ASP.NET enterprise app for the last year. This is a business-critical system with high impact on the bottom line. Around 3000 users. It will replace a mainframe app which costs us several million dollars per year in maintenance fees. It consists of 8 solutions, each solution contains 4 projects, and there will be almost 300 web pages and 150 Cognos reports when we are done. The solutions are aligned along business processes, i.e. money management, customer management, etc. One solution contains modules and datasets commonly used by multiple other solutions, and another solution contains just the 'back-office' screens which are used to edit master files and company setups.
We've found the following to be helpful: 1) Use Visual Source Safe as the central code repository. Visual Studio integrates VSS fairly well (but not perfectly). A caveat - do not use VSS Explorer to check in/check out files once Visual Studio has added them to VSS or checked them out. Otherwise you WILL get out-of-sync if you are careless, and someone's code will be overwritten. 2) Use the Partitioned Single Solution model (remember we are talking about large systems here). We tried the Multi-Solution model but it was very troublesome due to file references getting frequently broken. For us this model gives the best mix of separation of developers and centralization of code. You can use the Single Solution model if you don't mind the long load and Rebuild times in Visual Studio. 3) Every developer has his own web server to develop on. When development is complete, the code is checked into VSS. Code does not get checked in unless it compiles. 4) Twice daily a separate Build Server uses a batch file to GET LATEST from VSS (using a Single Solution which contains all 32 projects) and Rebuild the whole application. If the build fails the batch file sends an email to me and the responsible developer is flogged. If the build succeeds the resulting asp.net web pages and DLLs are XCOPY'd to the development department web farm (2 servers). 5) Our QA people test the code on the development web farm. The individual developers do not have access to these web servers. 6) During our periodic user acceptance testing the candidate build is copied to a testing web farm (2 servers). There the business owners can test the code without being affected by ongoing developer work.
Key learnings: 1) Do not use file references. Run away as fast as you can. 2) Every developer has their local solutions and projects in the same place on every computer. This simplifies VSS work and removes the need to change paths. 3) Don't mess with the Reference Path property page in Visual Studio. Let Visual Studio figure it out; if every developer has the same project structure in the same place on every computer then VS and VSS will work well together. 4) Strike a balance between the number of solutions, and the number of projects in each solution, and the need for multiple developers to work on the same projects. We aligned the solutions along business process lines but some other arrangement may work better for you. The point is to prevent developer 1 from waiting on developer 2 to check in a project file.
Hope this helps. Good Luck with your app.
Jeff K.
<snip>
>To repeat my question, how does one structure multiple solutions (with >multiple projects each), to best facilitate development and testing? [quoted text clipped - 5 lines] > >. michaelj - 11 Dec 2003 02:05 GMT Thank you Jeff for responding. There has been some progress since my first post.
Our application currently has 10 solutions with roughly 60 projects spread between them. We have managed code projects , some mixed mode DLL's, and a fair amount of unmanaged legacy code that we need to leverage. We are about a third of the way to our first release and we expect these numbers to double before we are done. Our application also has high visibility with our customer base (5000 users) and it figures prominently in our bottom line. This is third generation development and, as a result, we have had plenty of opportunity to consider questions of design. In this at least we, are confident. We release on a short cycle (every 6 months) so minimizing testing by understanding and limiting dependencies is very important to us. There are only three developers working on this project but we are seasoned veterans; that, and they work us near to death. :)
Our project is too large to effectively build on developers machines. As I mentioned in my first post, we have resorted to installing, via our installer, and then building just the components we are working on. This means that using file references is mandatory, but to be honest it really hasn't been an issue. We use project references between all projects that reside in the same solution, and file references for all external dependencies . We are using batch files in post build steps to copy DLL, EXE, and PDB files to the install location. It would be nice if the compiler would just put the outputs where we want them, but c'est la vie.
We are also getting a handle on SourceSafe integration. In some respects it is better, but on the whole I would say that it is a step backwards. My biggest complaint is the reference path. Apparently it is established when the reference is made and it is stored in the project as a hint path. This is like getting directions from my grandpa, you might get to where you are going, but if you haven't been there before, you will never be sure. Using hints to resolve type dependencies is not my idea of a quality development process. Add to this that the references are stored in user files that are not source controlled and we have a real mess in the making. I wonder if they might invent a few more places to store binding information.
We are looking at an external tool (Visual Build) to run our build machine. It looks fairly promising but getting so that we can do a fresh get from SourceSafe and build into a clean folder has been difficult. As you suggested, we mandate that developer file structures match those in SourceSafe.
Overall I am hugely disappointed in Visual Studio and it's capabilities for enterprise development. What idiot invented this copy local flag for references? It only copies managed binary dependencies, it doesn't copy other content files, and setting it to false causes the development environment to crash. It seems applicable to only the most trivial of development efforts.
If there were a way to market all that I have learned, I might very well be rich. On the whole I have concluded that non-trivial development is possible, however, in my opinion, only the most experienced of developers are going to succeed.
Michael (mjj0000) at (yahoo) dot (com)
> HI, > > My team of 6 developers has been working on a multi- > solution, multi-project ASP.NET enterprise app for the > last year. This is a business-critical system with high ><snip>
> 2 to check in a project file. > [quoted text clipped - 16 lines] > > > >. Jeff K - 24 Dec 2003 20:45 GMT Sorry to be so long in answering, I've been out of town.
Trust me, getting rid of file references will help a lot. I may be mistaken but that local copy issue goes away when you use Project references throughout.
I understand your developer machines cannot build the entire system, so if you can designate 1 PC or server for building you can use the Source Safe commandline utility to Get Latest in a .cmd file, then call the compiler in the same batch file. To Get Latest just do this:
ss Get $/CCM/Clients/DM2/CCMApp -R
Where the path after $/ matches your VSS project structure. Of course set the current drive and folder to the location of ss.exe on the build server. We get a master solution that contains all of the projects contained in the other solutions, by doing this no file references are necessary.
To build the solution just do this: (this should be on one line)
devenv c:\projects\MasterSoln\mastersoln.sln /nologo /rebuild
Of course set the current drive and path to the location of devenv.exe. I check the return values and use other batch code to email a success/fail message to me so I can run this as a scheduled task on a server in another building.
If every developer has the same solution/project structure, and the same Working Folder path in VSS, I think the reference path issue will be moot.
Maybe it's just the differences in our implementations but I've found VS.NET much improved for enterprise/multi- developer authoring.
Anyway I understand your pain having had some of the same problems early on. Good Luck to you...
Jeff K.
>Our application currently has 10 solutions with roughly >60 projects spread > snip Don McNamara - 11 Dec 2003 15:48 GMT Hi Michael,
We have a decent size project going here... about 300k lines of code, 6 client applications and lots of seperate assemblies and web services -- also database projects. We went assembly crazy at the beginning (we have over 100 asseblies) but, we are slowly consolidating to reduce reference complexity. Also, there are about 25 - 30 developers. And I'm guessing about 2000 source files. We have major production releases 4 times a year.
We use a home grown build tool and it has been meeting our needs fairly well. We use Source Safe for all work -- it has been working, but I can't wait for the next version. We are running daily builds on a dedicated build machine -- currently we are building every hour.
Essentially, the build tool follows the build process outlined in the MS document you referenced. It was non-trivial to write due to the rather convoluted Visual Studio project and solutions files. Our build tool does a get from source safe, parses all the project files and generates a makefile that calls the compilers (csc, vbc) directly. The makefile is generated so all interdependencies are resolved and built in the correct order. We do delta builds hourly during the business day, only building what has changed and any assemblies that depend on those, etc. We do full builds twice a day.
For simple (one assembly) local development we use a common share point for references - this share point is deployed to on an as needed basis using the output of the build tool. For complex development (several interconnected assemblies) the developer runs local delta builds on his or her workstation. The developer repoints the Reference Path to the locally built assemblies. We have had very little problem with the Reference Path settings -- having them in a non-source controlled file has helped more than hurt us.
I agree with matching Source Safe and the local file structure. One thing that has also been handy for us is to match the path to the assembly root namespace.
Things that have been a sore point for us: 1. Breaking the build -> it is easy for a developer to check in code that does not compile. With hourly builds we find these problems sooner rather than later. 2. Concurrent development on multiple releases -- we have not found an easy way to do this with source safe.
Things I might change: 1. Build Tool -- it is very complex. It might have been better to investigate something along the lines of nant. 2. Database projects -- they seem to be the step-child of the Visual Studio family. It might have been better to look at 3rd party solutions here. 3. The sheer number of our assemblies. This increased the complexity significantly. One class per assembly is not good design. We are looking at consolidating core assemblies on the client side and server side.
I would strongly suggest reading Martin Fowler's article title Continous Integration: http://www.martinfowler.com/articles/continuousIntegration.html
I'm not sure how much help this posting is. Please let me know if you have any questions or if you want me to go into detail on anything specific.
Also, you might want to look at the Whidbey release of Visual Studio. Its release still a bit off in the future, but it might steer some of your decisions.
Good luck, Don
> My questions are directed to those who are successfully doing non-trivial > "enterprise" development using Visual Studio 2003 .NET. It has been my [quoted text clipped - 55 lines] > > Michael
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|