Project Evaluation

Development Process:

Our product has evolved from it's initial concepts presented to us by Hotel 451, our customer group, into a full fledged hotel management system. To implement such a large project we used a modified waterfall development scheme as described at func/team_structure.html.

During the analysis and specification of the project we were presented with a complex problem revolving around the proper modeling of the data. To solve this we developed our model using standard DFDs and ERDs which allowed us to model the movement and usage of data within the system. Upon completing the initial design of the data stores we attempted to determine key algorithmic points within the program. These key points were further studied, in light of the data model, so that we could determine the most appropriate algorithm for each case. These included the algorithm used to calculate free rooms for any given day, and the algorithm for "combing" the database for pending room service requests. Upon completion of the algorithmic designs we were ready to develop the screen mockups.

These screeen mockups were of extreme importance, especially when one considers that the project was evaluated not only by the customers, but also by an outside group that was not familiar with all the problems and requests for with the project. Once the screen mockups were completed it was necessary, Due to the design methodology we were following, to design the user manual. As specified this was a minimalist manual containing the base level of information required to effectively operate the software. The next stage of development evolved to coding of the project.

In the end the programmingof the project took the majority of the last month. To some degree it was a scramble to successfully complete the requested product. Due to the major effort required to create the project some portions of testing were neglected. This neglect was limited to performance testing and implementation of our correction and monitoring procedures. As it was most important all function level testing was completed during the later portion of the development phase. Our reason for not following the testing procedures we originally specified was our lack of time to follow the detailed levels of management required for our original procedures. In the end testers reported their bugs directly to the programming group. We found, however that this did not generally create any problems. Perhaps our original plan contained too many levels of management to be practical in this size of project.

Satisfying of Requirements:

Our customers gave us a list of requirements that were lacking several important details. (please see our customers "Informal Specification of Requirements" document for list of requirements) Due to this lack of information several meetings were required to further develop the conceptual model of the product. One failing that was perceived late in the development was our lack of any written modifications to the initial design concept. Although verbal communication did not present any difficulties in the beginning of the project our attempts to remember what was originally said to us led us to reconsult the customer group to ensure that we were completing the project to their satisfaction. We feel that we have met the requirements of the system with the exception of proven network interoperability. However, we did not feel that we needed to prove that networking would work in the trial version since our product was based on an architecture and database that are well known to support networking.

Features added or removed over time:

We did not feel that the customer group placed any unrealistic problems on the table before us. Therefor, we did not have a need to remove or add anything to the program over time. We were primarily required to modify the functionality of several items, rather than adding or deleting them in entirety.

Lessons Learned:

1. We originally decided not to use an HTML editor, instead HTML was done by hand. This proved to be very time expensive, we would therefor recommend against hand written HTML in future projects. Several other groups appear to have had good success with the Netscape HTML editor. It may be a useful tool in future design work.
2. Initially we found that the programmers did not follow our original color or interface designs. Other subgroups also did not seem to interact as well as was expected. We feel that one member from each major team should be placed on one other team. This would facilitate inter-team communication. Such a level of interaction would have allowed us to catch and fix our interface problems at a much earlier stage.
3. Selection of a language that was not freely available to all group members was probably a bad choice. In a project of this nature, where time is in short supply it is important that everyone in the group have access to, and be as proficient in the chosen language as possible. It severely limited who could write code for our project when we chose a language that only a few of the group member knew, or owned.
4. Tighter organization of tasks and better enforcement of meeting attendance is a must for a project of this nature. More often than not portions of the assignment were left for the clean up crew or were not started until the day the document was assembled. This prevented a good consistency, spelling, and grammar checking of the assignment. The quality of the document would be greatly improved if all the portions were written before assembly of the document's final draft started.
5. Small problems tended to grow. This ties into point number two as a problem stays in the project, over time the magnitude of effort required to correct the change also increases. It is far more important to remove problems at and earlier stage than to attempt to fix them later in order to have greater progress in the design.
6. Ratification of interface was left far too late in the process. Part of our difficulties with our interface stem from the fact that the design itself was in flux through a portion of the coding phase, and throughout the design phase of the system. Creating the correct interface is easily as important as creating a good data model; possibly it is more important than creating a good algorithmic structure with in the program. More attention should have been paid to the creation and ratification of the design at an earlier stage because it is very hard for the programmers to hit a moving target, which is was our interface really was.
7. Lack of detail and misunderstanding of project goals and scope plagued our initial analysis phase. It would have been far more advantageous to have spent more time clearly defining the utility of the program than to have placed more effort into the initial analysis of the system. Ambiguity in the design goals, caused by an unfocused concept of the problem, led to fluctuations in the algorithm designs as each person attempted to design their assigned algorithms within the context of differing interpretations of the design goals. A written statement of the exact (or more exact) goals of the consumer would have helped keep a tighter philosophical cohesion during the analysis and algorithmic design stages.
8. A group must find a group leader as soon as possible with several very important characteristics. This person must be able to attend all meetings, so that they can act as a liaison between all the separate subgroups. The leader requires excellent skills in planning ahead for and remembering the deadlines within the course. This person must also have an overwhelming desire to produce the best product they possibly can, any other sort of person will tend to let things slowly slide into chaos. They are required to be pushy to an extent; motivation of group members is not an easy task, especially so for someone who is not forceful in their interactions with people. Finally they must have the ability to asses the strengths and weaknesses of each group member, assigning tasks accordingly. We felt that Clinton performed this role quite well, although at times he was perhaps not forceful enough in his motivation of the group.
9. Preparation of the presentation far ahead of time seems to have made the difference between a well polished presentation and an adequate one. Familiarity with the flow of the presentation, and simple practice has helped our group and many others to appear as professional as possible. Early preparation and practice also allows the group to asses the presentations weak and strong points. Several other groups seemed to lack a knowledge of their weak points, leading them to either embarrass themselves when their product failed, or to be so nervous as to show fear to the audience - a horrible mistake during a presentatoin.
10. We found that customers are not very good at defining their requirements. More interaction with the customer group is required in order to avoid costly changes at later stages. To this end we would suggest that groups schedule extra time outside of class to further define the initial application.
11. Another lesson dealt with deletion. Many customers don't know what data is really valuable to them. They are, therefor, often confused as to whether they should be able to delete data from their database, what types of data they should be able to delete; and what sort of action should be taken if true deletion is not advisable. It was late in the development cycle that our customers decided that they wanted to keep all the possible data in the system. We were therefor forced to rework our storage schemes for the product. A group should ensure that they have all the data management requirements, such as what to keep, when to keep it and so forth, as soon as possible in a project of this nature.
12. An important lesson that we learned is the problem of the supplier-costumer interaction. The supplier group always needed more information about the system, but like in the real world, the costumer group never had a clear idea of their real needs. We also had time meeting problems, the costumer group wasn't available in the time in which we wanted to meet them, and as a concequence the supplier team had few contact with our costumer. But this is the way in which real projects work, its very hard to find a good costumer who would be always available to answer all the questions.

Changes for next time:

1. We would wish to enforce much more strict document construction guidelines within our HTML pages. In the current version of our documents there are many subtle differences in writing style and layout that destroy the overall cohesion of the documents. A published guide to page layout and writing style would have helped to avoid the reformatting headaches that occurred in the first three documents.
2. Picking a language we all have access to and are proficient in would have greatly aided our design and coding phases. All members would understand the limitations of the language and would be able to fix bugs on their own rather than relying on a select group to do so. These two facts alone almost offset the time cost involved in picking a language such as C++ over Microsoft Access. In the future we would recommend an emphasis on a language common to all members of the group. This is not really necessary in a real business situation as there would be time for people to learn the new language as they work with it full time. However, in our time limited situation it is necessary that all group members are familiar with the language from the outset.
3. Enforcing more stringent time restraints on all aspects of the development effort. In many instances people did not have their portion of the work done at the required time thereby delaying and complicating the completion of that aspect of the project. A method must be found that would encourage timely completion of work with out the threat of an Iron Ruler. In the real world this function is played by money. Many students are only concerned with passing a course, not with the actual grade that they receive.
4. Tighter focus on the final result of the project is also required. Due to the immense size of some portions of our project and the tight time constraints that surround them focus was often shifted to a single minded goal of producing the required volume of documentation. This hurt the overall project by tempting authors to make certain sacrifices to the overall project for short term gain within the documentation.
5. Greater detail should have been imposed on the interface at a much earlier stage in development. The interface should not be fluctuating very much towards the beginning of the coding phase. It should be firmly fixed by the time the process of development has progressed past the point of implementing the code that will underlie the interface.
6. We would like to get far more detailed requirements at an earlier stage, in writing if possible. As well, we would have preferred that the customer group submit all the changes they requested in writing. It was often difficult to remember verbally exchanged changes to the system. More than once peoples memories differed on certain points, causing us to refer back to the customer group.
7. Completion of the coding segment at least 3 weeks before the presentation. This is more of an unattainable goal to shoot for rather than a hard and fast goal. It would have allowed us more time in debugging than we actually had. Although our debugging was more or less complete the extra time would have allowed us to add further enhancements to the product. We would also have felt more confident at the end if we knew that we had had extra time to apply to the project. It seemed as if the project expanded to fill all the time we had available for it.
8. Choice of presenters is especially important in this class as a major portion of our overall grade depends upon the presentation itself. We would have preferred to audition our presenters to ensure that they had as good a public speaking skills as they claimed to have had. On the whole however we were very lucky with the skills that our presenters had, however, some members of the group were still nervous about the choice of presenters before seeing them in action.
9. Change the schedule for the completion of the user Manual. The date for the completion of the manual should be changed because after the user manual was completed the coding part of the project continued. As a concequence there were alot of changes in the way many functions worked in the original system. This changes caused that the user manual was less then helpful for the use of the final program.

Suggestions for future course adjustments:

1. We found that there was little extra time to properly change our user manual as we encountered coding problems and thought of several new improvements to our algorithms and our interfaces. We were unable to implement them due to the fact that there was not going to be enough time to ensure that the user manual was updated properly. Therefor, were forced to use our user manual as a blue print for the design and implementation of the code; regardless of any enhancements that could have been added. Perhaps the development of a much shorter user manual to be accompanied by a more complete final manual later would allow for more substantial changes to the algorithms and interfaces as the coding process matures.
2. We found groups with skills in Visual Basic, Delphi, and Powerbuilder had a distinct advantage. There was little use of C++, Sybase, etc., in any of the products demonstrated. It seems rather pointless to create groups based on experience in unusable skills. We would suggest arranging the groups around knowledge of one of the 3 rapid application development tools listed above. All the other skills listed at the beginning of the course were essentially of no relevence during the development of our product.
3. The University of Calgary Computer Science department has few of the main tools used in development our applications. All the projects revolve around rapid development of databases. The main industry tools for such tasks are Visual Basic, Delphi and Powerbuilder, which one would expect to be used in a course that strives to model the real world. However, we were limited to the use of tools that we either were willing to purchase or that members of the group already possessed. This seemed to not only limit our choices for development tools, but to also give an unfair advantage to groups that happened to posses Visual Basic or Delphi. The tools used in the class should either be restricted to tools that are available to all students, or should be provided in some manner for students of this class. More than likely Microsoft and Borland would be more than willing to provide their tools at a substantial discount for a course of this nature.
4. In many real life situations one would not use such a formal presentation method as was used in the final presentation. It transmits far less useful information that an small group hands on meeting would. We therefor feel that too great an emphasis was placed on the final presentation. In its place we would suggest a smaller hands on meeting involving the professor, the TAs and the customer group. The product could then be explored in much greater detail by the customer group.
5. The volume of documentation requested for this course prohibited the production of any truly high quality documents. We were concentrating so heavily on producing the required number of pages that often more important aspects such as grammatical and stylistic construction took a back seat. Less emphasis on quantity (i.e., remove the 80 page minimum) and a greater emphasis on a clear, concise, and cohesive documentation seems more appropriate for this course. We are attempting to produce an easy to use product, which by definition should be simple to understand, and therefor should have very clear documentation to accompany it.
6. Another suggestion would be to combine the two design documents. This would allow students to place more time on design refinement. It seemed that there was almost no changes requested after the Overall design document. One would also suggest that the combined design document be followed up by a short letter detailing acknowledgment of changes and requests for clarification. In this way we could accomplish the same thing as the current two documents with far less paper work.
7. The course seemed to contradict itself in a very disturbing way. Projects were constructed following an older waterfall methodology while lectures dealt almost exclusively with more modern object oriented design philosophies. It seemed odd that the course should be instructed in one methodology while being required to implement another. We would suggest a greater consolidation between the design philosophies of the project and that of the lectures.

Testing:

For the most part our testing procedures were implemented as originally specified. However, the method of implementation did not entirely follow the scheme we originally had envisioned. Our function level testing seemed to have been performed by the group as a whole rather than a specific group of testers. We also found that it was far too costly in terms of time to add a layer of management to the bug testing and fixing cycle. To this end, we therefor reported bugs directly to the coders who then repaired the bugs, often on the spot and at the time of report. In the future we would like to suggest that other groups pair a tester or a set of testers with a programmer. In this fashion one is able to debug and fix in real time, a definitive time saver.

We originally had planed to implement an overall consistency check as a final test for the product. In the end we abandoned this idea for a couple of reasons. First of all it would have been very time consuming for someone to design on pen and paper a set of tests that would exhaustively test the system. Secondly, we found that function level testing took care of our consistency check for us. This was especially true when we added screen operations to our list of functionality to test at the function level. We constructed the lowest level functionality first, which was then followed by more complex functionality. For example, the add, modify and delete room functionality was designed and built before the management screen functionality. Since we were testing each piece as it was built the lower level functionality was assured to work while the next level above it was being implemented. It was not necessary to test anything more complex on the management screen (from our example above) than the linkage between it and it's lower level functions. In effect we substituted a bottom up testing approach for a top down approach. In the end we believe it saved us time because it was easier to individually organize a small number of tests for a subsection than it would have been to organize and manage a full system wide top down testing schedule.

Group Structure:

Group structure did not end up anything like we had originally envisioned. Almost everyone, except for Brian and Fan our programmers, ended up working of the documentation. So we infact needed close to 10 people instead of the six originally decided upon. This was due to the tight time constraints that surrounded the larger two documents. Initially our idea of using 5 people to design the software worked well. A group of three of us created the data flow and algorithm designs. However, when it came time to create the user interfaces it seemed that everyone wanted to participate in that portion of the design phase. The old saying too many cooks spoil the brew is very true in this case. We ended up spending a large portion of the time arguing over less important details such as coloring of the screens. In an initial model of the system it is really not worth while to bother with such things. They are better left to the final release version of the product, as long as you assign someone to take care of them.

Our original concept of the programming group required for the project was very close to what we actually implemented. With the exception of having only 2 full time programmers, and a part time person doing work on interface clean up and redesign it seems to have followed our team strucutre very closely. Our testing was not as close to our first model as we would have liked. We ended up having everyone doing some amount of testing during the final couple of weeks of the project. As mentioned earlier we also dispensed with the idea of having a buffering layer of management between the programmers and the testers.

On the whole we do not feel that we had a good enough grasp of what was needed in this project before we began it to have accurately planed out a strong management or team structure. In the future we would prefer to have the ability to analyize the requested for a product for a period of time before having to put a mangement plan in place. It might also have been easier to create a set of 2-3 person teams that had a good strong cross section of skills so that they could be moved around dynamically as the project changed.

Problems Noted By Individual Group Members:

Note: the quotations expressed below are the sentiments of individual people and should not be attributed to the group as a whole. Due to the nature of these quotations names have been removed.

Individuals doing what they wanted and hiding it or not mentioning it to the group as a whole. Especially individuals who did not follow the guidelines established for the user manual.

Group members enthusiasm and participation dwindled off near the end. The majority of the work was left to a small group of people.

Several group members carried more weight.

Visual Basic was not a good language due to the need for all the .dlls. It therefor didn't run well on several peoples computers.

Customer group feedback was poor. No matter how bad our documents were they were accepted by the customer group.

Customers changed their minds late in development or forgot what they originally requested.

Customers never set out a consistent set of requirements. There were many loop holes and too much guess work as to how they wanted things to work.