10.46243/jst.2022.v7.i02.pp395-422 registered
A Survey of Controlled Experiments in Software Engineering
Resolves to https://www.jst.org.in/index.php/pub/article/view/559
Held by Longman Publishers (India) · prefix 10.46243 live · DOI address https://doi.org/10.46243/jst.2022.v7.i02.pp395-422
Registered 29 Sep 2026 via crossref · record version 2 · last change 29 Sep 2026, 11:59 PM · record sha256 91df6658a7d08d4f…
Resolve ⬇ Record (JSON) ⬇ Kernel Metadata Declaration (XML) Compare with Crossref Cite (APA · BibTeX · RIS · CSL)
Also as schema.org JSON-LD · Turtle · RDF/XML · DataCite XML · Crossref UNIXREF · Crossref UNIXSD — the formats of the DOI content-negotiation service, from this record (which carries what).
What the DOI identifies
JournalArticle — an article in a journal · Digital · Visual · en
A Survey of Controlled Experiments in Software Engineering (PrincipalTitle · en)
Published 2022-07-03
Part of Journal of Science & Technology · ISSN 2456-5660 · volume 7 · issue 2 · pages 395–422
Agents
- Dr.M.V Siva Prasad Dr.M.V Siva Prasad (author)
- Longman Publishers (publisher)
Identifiers DOI 10.46243/jst.2022.v7.i02.pp395-422
Abstract · en
The classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant information is reported. Among the 5,453 scientific articles published in 12 leading software engineering journals and conferences inthe decade from 1993 to 2002, 103 articles (1.9 percent) reported controlled experiments in which individuals or teams performed one or more software engineering tasks. This survey quantitatively characterizes the topics of the experiments and their subjects (number of subjects, students versus professionals, recruitment, and rewards for participation), tasks (type of task, duration, and type and size of application) and environments (location, development tools). Furthermore, the survey reports on how internal and external validity is addressed and the extent to which experiments are replicated. The gathered data reflects the relevance of software engineering experiments to industrial practice and the scientific maturity of software engineering research
System metadata — ISO 26324:2025, Annex B · DOI Handbook 10.1
Each element by the standard's name (Annex B: reference elements, then administrative) and the Handbook's (in grey), read off the record above.
| Element | Value | In the record |
|---|---|---|
| DOI Name DOI name | 10.46243/jst.2022.v7.i02.pp395-422 | doi |
| Referent Type referentType | Creation | referent |
| Referent Sub-Type referentSubType | JournalArticle — an article in a journal | type |
| Referent Name(s) referentName(s) | A Survey of Controlled Experiments in Software Engineering (PrincipalTitle, en) | titles |
| Basic Metadata basicMetadata | author: Dr.M.V Siva Prasad Dr.M.V Siva Prasad publisher: Longman Publishers published: 2022-07-03 part of: Journal of Science & Technology · ISSN 2456-5660 · vol. 7 · no. 2 · pp. 395–422 language: en form: Digital · Visual · Language | agents, dates, container, language, structural_type, modes, characters |
| Referent Identifier(s) alternateIdentifier(s) | none besides the DOI | identifiers, relations (IsSameAs) |
| Registration Authority registrationAuthorityCode | Crossref — issued by Crossref (member 25296); held here as a copy | record.source_agency (our code, ra_doi_name, for names issued here once appointed) |
| Created Date issueDate | 2024-02-16 | record.registered (when the DOI name was first registered) |
| relatedIdentifiers | none needed — the descriptive metadata is in this record | container, relations (only where the descriptive metadata lives at another identifier) |
complete Every System Metadata element is here, with the basic metadata a journal article needs.
The System Metadata Declaration (JSON) · the Kernel Metadata Declaration (XML) · what each sub-type needs
Cited by — in this registry
No record held in this registry cites this DOI.
Counts only records held in this registry; a citation from a work held elsewhere is not seen here. The list as JSON
References with DOIs
The record lists 52 references; 41 carry a DOI.
| # | Reference | DOI | Held here |
|---|---|---|---|
| 3 | ref3 E. Arisholm and D.I.K. Sjøberg, “Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software,” IEEE Trans. Software En… | 10.1109/tse.2004.43 | not held here |
| 4 | ref4 V.R. Basili, “The Experimental Paradigm in Software Engineer- ing,” Experimental Eng. Issues: Critical Assessment and Future Directions, Proc. Int’l Workshop, vol. 706, pp. 3-12, … | 10.1007/3-540-57092-6_91 | not held here |
| 5 | ref5 V.R. Basili, “The Role of Experimentation in Software Engineering: Past, Current, and Future,” Proc. 18th Int’l Conf. Software Eng., pp. 442- 449, 1996 | 10.1109/icse.1996.493439 | not held here |
| 6 | ref6 V.R. Basili, R.W. Selby, and D.H. Hutchens, “Experimentation in Software Engineering,” IEEE Trans. Software Eng., pp. 733-743, July 1986 | 10.1109/tse.1986.6312975 | not held here |
| 7 | ref7 V.R. Basili, F. Shull, and F. Lanubile, “Building Knowledge through Families of Experiments,” IEEE Trans. Software Eng., vol. 25, pp. 456- 473, July/Aug. 1999 | 10.1109/32.799939 | not held here |
| 8 | ref8 D.M. Berry and W.F. Tichy, “Response to ‘Comments on Formal Methods Application: An Empirical Tale of Software Develop- ment,’” IEEE Trans. Software Eng., vol. 29, no. 6, pp. 572-… | 10.1109/tse.2003.1205183 | not held here |
| 9 | ref9 D.T. Campbell and J.C. Stanley, “Experimental and Quasi-Experimental Designs for Research on Teaching,” Handbook of Research on Teaching, N.L. Cage, ed., Chicago: Rand McNally, 19… | 10.1177/001316446702700247 | not held here |
| 12 | ref12 B. Curtis, “Measurement and Experimentation in Software Engineering,” Proc. IEEE, vol. 68, no. 9, pp. 1144-1157, Sept. 1980 | 10.1109/proc.1980.11813 | not held here |
| 14 | ref14 I.S. Deligiannis, M. Shepperd, S. Webster, and M. Roumeliotis, “A Review of Experimental Investigations into Object-Oriented Technology,” Empirical Software Eng., vol. 7, no. 3, p… | 10.1023/a:1016392131540 | not held here |
| 15 | ref15 A. Endres and D. Rombach, A Handbook of Software and Systems Engineering: Empirical Observations, Laws, and Theories, Fraunhofer IESE series on software engineering. Pearson Educa… | 10.1002/0471028959.sof135 | not held here |
| 16 | ref16 N. Fenton, “How Effective Are Software Engineering Methods?” J. Systems and Software, vol. 22, no. 2, pp. 141-146, 1993 | 10.1016/0164-1212(93)90092-c | not held here |
| 17 | ref17 R. Ferber, “Editorial: Research by Convenience,” J. Consumer Research, vol. 4, pp. 57-58, June 1977 | 10.1086/208679 | not held here |
| 18 | ref18 R.L. Glass and T.Y. Chen, “An Assessment of Systems and Software Engineering Scholars and Institutions (1998-2002),” J. Systems and Software, vol. 68, no. 1, pp. 77-84, 2003 | 10.1016/s0164-1212(03)00232-2 | not held here |
| 19 | ref19 R.L. Glass, V. Ramesh, and I. Vessey, “An Analysis of Research in Computing Disciplines,” Comm. ACM, vol. 47, no. 6, pp. 89-94, June 2004 | 10.1145/990680.990686 | not held here |
| 20 | ref20 R.L. Glass, I. Vessey, and V. Ramesh, “Research in Software Engineering: An Analysis of the Literature,” J. Information and Software Technology, vol. 44, no. 8, pp. 491-506, June … | 10.1016/s0950-5849(02)00049-6 | not held here |
| 21 | ref21 S. Greenland, J.M. Robins, and J. Pearl, “Confounding and Collapsibility in Causal Inference,” Statistical Science, vol. 14, no. 1, pp. 29-46, 1999 | 10.1214/ss/1009211805 | not held here |
| 22 | ref22 W. Hayes, “Research Synthesis in Software Engineering: A Case for Meta-Analysis,” Proc. Sixth Int’l Symp. Software Metrics, pp. 143- 151, 2003 | 10.1109/metric.1999.809735 | not held here |
| 23 | ref23 M. Ho¨ st, C. Wohlin, and T. Thelin, “Experimental Context Classification: Incentives and Experience of Subjects,” Proc. 27th Int’l Conf. Software Eng., pp. 470-478, 2005 | 10.1109/icse.2005.1553590 | not held here |
| 25 | ref25 M. Jørgensen, “A Review of Studies on Expert Estimation of Software Development Effort,” J. Systems and Software, vol. 70, nos. 1,2, pp. 37-60, 2004 | 10.1016/s0164-1212(02)00156-5 | not held here |
| 26 | ref26 M. Jørgensen and D.I.K. Sjøberg, “Generalization and Theory Building in Software Engineering Research,” Empirical Assessment in Software Eng. Proc., pp. 29-36, 2004 | 10.1049/ic:20040396 | not held here |
| 27 | ref27 M. Jørgensen, K.H. Teigen, and K. Moløkken, “Better Sure than Safe? Over-Confidence in Judgement Based Software Develop- ment Effort Prediction Intervals,” J. Systems and Software… | 10.1016/s0164-1212(02)00160-7 | not held here |
| 28 | ref28 N. Juristo, A.M. Moreno, and S. Vegas, “Reviewing 25 Years of Testing Technique Experiments,” Empirical Software Eng., vol. 9, pp. 7-44, Mar. 2004 | 10.1023/b:emse.0000013513.48963.1b | not held here |
| 30 | ref30 B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K. El-Emam, and J. Rosenberg, “Preliminary Guidelines for Empirical Research in Software Engineering,” IEEE… | 10.1109/tse.2002.1027796 | not held here |
| 31 | ref31 R.M. Lindsay and A.S.C. Ehrenberg, “The Design of Replicated Studies,” The Am. Statistician, vol. 47, pp. 217-228, Aug. 1993 | 10.1080/00031305.1993.10475983 | not held here |
| 32 | ref32 C. Lott and D. Rombach, “Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques,” Em- pirical Software Eng., vol. 1, pp. 241-277, 1996 | 10.1007/bf00127447 | not held here |
| 33 | ref33 J.W. Lucas, “Theory-Testing, Generalization, and the Problem of External Validity,” Sociological Theory, vol. 21, pp. 236-253, Sept. 2003 | 10.1111/1467-9558.00187 | not held here |
| 34 | ref34 T.R. Lunsford and B.R. Lunsford, “The Research Sample, Part I: Sampling,” J. Prosthetics and Orthotics, vol. 7, pp. 105-112, 1995 | 10.1097/00008526-199500730-00008 | not held here |
| 35 | ref35 Experimental Software Engineering Issues: Critical Assessment and Future Directions, Int’l Workshop Dagstuhl Castle (Germany), Sept. 14- 18, 1992, Proc., H.D. Rombach, V.R. Basili… | 10.1007/3-540-57092-6 | not held here |
| 37 | ref37 W.R. Shadish, T.D. Cook, and D.T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Hought- on Mifflin, 2002 | 10.1086/345281 | not held here |
| 38 | ref38 M. Shaw, “Writing Good Software Engineering Research Papers: Minitutorial,” Proc. 25th Int’l Conf. Software Eng., pp. 726-736, 2003 | 10.1109/icse.2003.1201262 | not held here |
| 39 | ref39 D.I.K. Sjøberg, B. Anda, E. Arisholm, T. Dyba˚, M. Jørgensen, A. Karahasanovi´c, E. Koren, and M. Voka´ˇc, “Conducting Realistic Experiments in Software Engineering,” Proc. 18th I… | 10.1109/isese.2002.1166921 | not held here |
| 40 | ref40 D.I.K. Sjøberg, B. Anda, E. Arisholm, T. Dyba˚, M. Jørgensen, A. Karahasanovi´c, and M. Voka´ˇc, “Challenges and Recommenda- tions when Increasing the Realism of Controlled Softwa… | 10.1007/978-3-540-45143-3_3 | not held here |
| 41 | ref41 W.F. Tichy, “Should Computer Scientists Experiment More? 16 Excuses to Avoid Experimentation,” Computer, vol. 31, no. 5, pp. 32-40, May 1998 | 10.1109/2.675631 | not held here |
| 42 | ref42 W.F. Tichy, “Hints for Reviewing Empirical Work in Software Engineering,” Empirical Software Eng., vol. 5, no. 4, pp. 309-312, 2000 | 10.1023/a:1009844119158 | not held here |
| 43 | ref43 W.F. Tichy, P. Lukowicz, L. Prechelt, and E.A. Heinz, “Experi- mental Evaluation in Computer Science: A Quantitative Study,” J. Systems and Software, vol. 28, no. 1, pp. 9-18, Jan… | 10.1016/0164-1212(94)00111-y | not held here |
| 45 | ref45 R. Weber, “Editor’s Comments,” MIS Quarterly, vol. 27, no. 3, pp. iii-xii, Sept. 2003 | 10.2307/30036536 | not held here |
| 46 | ref46 C. Wohlin, P. Runeson, M. Ho¨ st, M.C. Ohlsson, B. Regnell, and A. Wesslen, Experimentation in Software Eng.: An Introduction. John Wiley & Sons Inc., 1999 | 10.1007/978-1-4615-4625-2_1 | not held here |
| 48 | ref48 E.A. Youngs, “Human Errors in Programming,” Int’l J. Man-Machine Studies, vol. 6, no. 3, pp. 361-376, 1974 | 10.1016/s0020-7373(74)80027-1 | not held here |
| 49 | ref49 M.V. Zelkowitz and D. Wallace, “Experimental Validation in Software Engineering,” J. Information and Software Technology, vol. 39, pp. 735- 743, 1997 | 10.1016/s0950-5849(97)00025-6 | not held here |
| 50 | ref50 M.V. Zelkowitz and D. Wallace, “Experimental Models for Validating Technology,” Theory and Practice of Object Systems, vol. 31, no. 5, pp. 23-31, May 1998 | 10.1109/2.675630 | not held here |
| 51 | ref51 A. Zendler, “A Preliminary Software Engineering Theory as Investigated by Published Experiments,” Empirical Software Eng., vol. 6, no. 2, pp. 161-180, 2001 | 10.1023/a:1011489321999 | not held here |
Relations
The record states no relation, and no record held here states one with this DOI.
Funding
The record names no funder.
Each funder's page lists every work held here that names it. ROR IDs open at ror.org; Funder Registry IDs resolve at doi.org.
History — the ledger
Every change to this DOI, in order, as it was recorded. Entries are only ever added, never changed or removed.
| # | When | What | By | Changes |
|---|---|---|---|---|
| 1 | 29 Sep 2026, 10:00 PM | register registered at Crossref; record read from api.crossref.org | Administrator (admin) | 199 fields set · sha256 8735a192e69d… |
| 2 | 29 Sep 2026, 11:59 PM | update record re-read from api.crossref.org | Administrator (admin) | agents.0.name.family: agents.0.name.given: container.titles.0.value: |
Machine-readable: the history as JSON, with the full record after each change.
Changes made at Crossref through DOI Doctor
Every deposit Smart Scholars' DOI Doctor made for this DOI, oldest first, with Crossref's own answer. Each fingerprint is the SHA-256 of its entry and of the fingerprint before it, so a change to any earlier entry changes every fingerprint after it.
| When | What changed | Crossref | Fingerprint |
|---|---|---|---|
| 11 Sep 2026, 05:18 | Address updated from https://jst.org.in/index.php/pub/article/view/559/506 to https://www.jst.org.in/index.php/pub/article/view/559 | ● accepted by Crossref | 1422326e6504… |
| 13 Sep 2026, 01:40 | Read back: the DOI resolves to the new address | ● confirmed | 55134cd32d97… |
| 07 Oct 2026, 07:37 | Metadata completed abstract (150 words) · licence | ● accepted by Crossref | 73856264b10d… |
| 07 Oct 2026, 08:48 | Read back: Crossref's record shows it | ● confirmed | e2f4b5e65d08… |
4 entries · the last fingerprint e2f4b5e65d08a94bb5addff442eee384c0efacffbacee1a4554b86c5e00dded8 · the history as JSON, each entry with the fingerprint before it, to check them yourself.



