Skip to main content

Assessing agile methods: An empirical study


Agile software processes emerged to address the issue of building software on time and within the planned budget. To adopt an agile process, it is imperative to analyze and evaluate its effectiveness in supporting high quality software development while complying with stringent time constraints. In this paper we describe an agile method for Web-based application development (XWebProcess) and an experiment conducted with a group of forty senior undergraduate students to assess the quality/speed effectiveness of the proposed method vis-à-vis the effectiveness of Extreme Programming (XP). The results have shown that the process proposed is equally agile when compared to XP, moreover, surveys conducted as part of the experiment pointed out that XWebProcess is more suitable to Web development in dimensions such as requirements gathering, user interface and navigation design, and software testing, therefore leading to better quality software.


  1. 1.

    Beck, K.Extreme Programming Explained. Addison-Wesley, 1999.

  2. 2.

    Jeffries, R., Anderson, A., and Hendrickson, C.Extreme Programming Installed. Addison-Wesley, 2001.

  3. 3.

    Sampaio, A.XWebProcess: An Agile Process for Web Application Development (In Portuguese). MSc Thesis, March 2004, Universidade Federal de Pernambuco.

  4. 4.

    Pressman, R.S.What a Tangled Web We Have. IEEE Software, January – February 2000, pp 18–21.

  5. 5.

    Object Management Group.Software Process Engineering Metamodel Specification. Version 1.0, November 2002.

  6. 6.

    Agile Manifesto Web Site. Available at:

  7. 7.

    Dynamic System Development Method (DSDM) Web Site. Available at:

  8. 8.

    Crystal Web Site. Available at:

  9. 9.

    Cockburn, A.Agile Software Development. The Agile Software Development Series, Addison Wesley, 2001.

  10. 10.

    Ambler, S.Different Projects Require Different Strategies. Available at:, 2002.

  11. 11.

    Lindval M., Basili, V., Boehm, B., et al.Empirical Findings in Agile Methods. Lecture Notes in computer Science, Volume 2418, pp 0197–0207, September 20th 2002.

  12. 12.

    Fowler, M.Refactoring: Improving the Design of Existing Code. Addison-Wesley, 1999.

  13. 13.

    Heinecke, H., Noack, C., and Schweizer, D.Constructing Agile Software Processes. XP2002 — Extreme Programming Conference, Available at:

  14. 14.

    Jacobson, I., Booch, G., and Rumbaugh, J.The Unified Software Development Process. Addison-Wesley, 1999.

  15. 15.

    Object-oriented Process, Environment and Notation (OPEN) Web site. Available at:

  16. 16.

    McDonald, A., and Welland, R.Agile Web Engineering (AWE) Process. Department of Computing Science Technical Report TR-2001-98, University of Glasgow, 2001.

  17. 17.

    Rumpe, B, and Schröder, A.Quantitative Survey on Extreme Programming Projects. In Third International Conference on Extreme Programming and Flexible Processes in Software Engineering, XP2002, May 26–30, Alghero, Italy, pp. 95–100, 2002.

  18. 18.

    Schwabe, D. and Rossi, G.The Object-Oriented Hypermedia Design Model. In Communications of the ACM, volume 38(8), pages 45–46, 1995.

  19. 19.

    Pastor, O., Fons, J., Abrahão, S., and Rámon, S.Object Oriented Conceptual Models for WEB Applications. In 4th Iberoamerican Workshop on Requirements Engineering and Software Environments (lIDEAS’2001), San Jose, Costa Rica, 2001.

  20. 20.

    Pfleeger, S. L.Design and Analysis in Software Engineering — Part1: The Language of Case Studies and Formal Experiments. Software Engineering Notes, vol 19, no 1, October 1994, pages 16–20.

    Article  Google Scholar 

  21. 21.

    Pfleeger, S. L.Experimental Design and Analysis in Software Engineering — Part2: How to Set Up an Experiment. Software Engineering Notes, vol 20, no 1, January 1995, pages 22–26.

    Article  Google Scholar 

  22. 22.

    Pfleeger, S. L.Experimental Design and Analysis in Software Engineering — Part3: Types of Experimental Design. Software Engineering Notes, vol 20, no 2, April 1995, pages 14–16.

    Article  Google Scholar 

  23. 23.

    Pfleeger, S. L.Experimental Design and Analysis in Software Engineering — Part4: Choosing an Experimental Design. Software Engineering Notes, vol 20, no 3, July 1995, pages 13–15.

    Article  Google Scholar 

  24. 24.

    Pfleeger, S. L.Experimental Design and Analysis in Software Engineering — Part5: Analyzing the Data. Software Engineering Notes, vol 20, no 5, December 1995, pages 14–17.

    Article  Google Scholar 

  25. 25.

    Wohlin, C., et al.Experimentation in Software Engineering: An Introduction. Kluwer Academic Puslishers, 2000.

  26. 26.

    Basili, V., et al.Experimentation in Software Engineering. IEEE Transactions on Software Engineering, Vol. SE-12, No. 7, July 1986.

  27. 27.

    Seaman, C.Qualitative Methods in Empirical Studies of Software Engineering. IEEE Transactions on Software Engineering, Vol. 25, No. 4, July/August 1999.

  28. 28.

    Sommerville, I.,Software Engineering, 6th Edition, Addison Wesley, 2003

  29. 29.

    Kappel, G. et al.Web Engineering — Old Wine in New Bottles?, In proc. of 4th Intl. Conference on Web Engineering, LNCS Vol. 3140, pages 6–11, 2004.

  30. 30.

    Lowe, D., and Eklund, J.,Client Needs and the Design Process in Web Projects, Journal of Web Engineering, vol. 1, p. 23–36, 2002, Rinton Press.

  31. 31.

    Wallace, D., Raggett, I., and Aufgang, J.,Extreme Programming for Web Projects (XP Series), Addison-Wesley, 2002.

  32. 32.

    XWebProcess website. Available at:≈atfs/xwebprocess.

  33. 33.

    Rombach, Dieter.The Dilemma of Software Development, Quality Connection, May 2002.

  34. 34.

    Szyperski, Clemens.Component Software: Beyond Object-Oriented Programming, Addison-Wesley, 2002.

  35. 35.

    Kleppe, A., Warmer, J., Bast, W.,MDA Explained: The Model Driven Architecture: Practice and Promise, Addison-Wesley, 2003.

  36. 36.

    Clements, P., and Northrop, L.,Software Product Lines: Practices and Patterns, Addison-Wesley, 2001.

  37. 37.

    Sampaio, Américo, Vasconcelos, Alexandre and Sampaio, Pedro R. Falcone,Towards Reconciling Quality and Agility in Web Application Development, In proc. of 1st. International Workshop on Web Quality — WQ’04, Munich, Germany, 2004 (to appear).

  38. 38.

    Sampaio, Américo, Vasconcelos, Alexandre and Sampaio, Pedro R. Falcone,XWebProcess: Agile Software Development for Web Applications, LNCS Proceedings of the International Conference on Web Engineering (ICWE 2004), pp 597–598 Munich, Germany, 2004.

Download references

Author information



Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Sampaio, A., Vasconcelos, A. & Sampaio, P.R.F. Assessing agile methods: An empirical study. J Braz Comp Soc 10, 21–48 (2004).

Download citation


  • Software Process
  • Agile Processes
  • XP
  • XWebProcess
  • Experimentation