ITEE Home » David Hyland-Wood » 2006SEPSurvey » 2006 Software Engineering Practices Survey - Summary of Results

Print version

Contents

1. Overview
2. Demographics
3. Geographics
4. Methodologies
5. Metrics
6. Requirements
7. Testing
8. Environment
9. Comments

Overview

Survey Title:	2006 Software Engineering Practices Survey
Purpose:	Determine popular tools, techniques and practices used by practicing software engineers.
Collection Approach:	Survey data was collected over the World Wide Web between 6 October and 6 November 2006. Respondents were identified by a "snowball" process. Direct email was sent to 139 software engineers personally known to me and also posted a link to the survey on my blog. Each invitee was asked to pass the invitation on to others.
Respondents:	448

Demographics

Education High School 44 Associate 6 3 year Bachelor 50 4 year Bachelor 191 Graduate Certificate 12 Masters 111 Post Masters 3 Ph.D. 30 Professional Doctorate 1 Total: 448

Employment by Organizational Type Educational 51 Government 19 Large Company 115 Medium Company 105 Small Company 86 Very Small Company 41 Self Employed 29 Unreported 2 Total: 448

Years of Experience
<1	6
1	11
2	13
3	18
4	21
5	25
6	40
7	29
8	29
9	23
10-12	82
13-15	31
16-20	49
21-25	37
>25	23
not reported	11
Total:	448

Number with industry certifications: 103 (23%)

Industry Certifications
sun	55
microsoft	31
ibm	14
oracle	5
cisco	4
novell	3
Scrum	3
Brain Bench	2
apple	2
ieee	2
pmi	2
redhat	2
cissp	2
bea	1
zend	1
peoplesoft	1
advagato.org	1
citrix	1
kepner-tregoe	1
rational	1
jack	1
lpic	1
genesys	1
cips	0
hp	0
iccp	0
juniper	0
mysql	0
nortel	0
omg	0
symantec	0
Total: 137

Number of current managers: 187 (42%)
NB: Some reported having managerial experience, but were not currently managers (8).

Years of Managerial Experience
<1	24
1	24
2	33
3	26
4	9
5	19
6	12
7	7
8	6
9	5
10-12	15
13-15	6
16-20	7
21-25	2
>25	0
Total:	195

Previously ISO 9000/9001 certified? 81
Currently ISO 9000/9001 certified? 80

ISO 9000/9001 by Organizational Type
Educational	7
Government	5
Large Company	32
Medium Company	25
Small Company	10
Very Small Company	0
Self Employed	0
Unreported	1
Total:	80

Number of organizations developing software primarily for their own internal use: 152 (34%)

Number of organizations developing software for commercial sale: 206 (46%)

Number of organizations producing software for other organizations (e.g. outsourcing, software services, contracting, consulting): 234 (52%)

Number contributing to Open Source Software (OSS): 202 (45%)

Employers doing OSS Large Company 51 Educational 35 Medium Company 34 Small Company 33 Very Small Company 23 Self Employed 20 Government 6 Unreported 0 Total: 202

US Employers doing OSS Large Company 44 Small Company 17 Educational 15 Medium Company 14 Very Small Company 11 Self Employed 10 Government 4 Unreported 0 Total: 115

Non-US Employers doing OSS Educational 20 Very Small Company 20 Small Company 16 Self Employed 12 Large Company 10 Medium Company 7 Government 2 Unreported 0 Total: 87

We were quite surprised to discover a difference in the demographics of Open Source Software creation. Large companies contribute more to OSS in the US than in the rest of the world.

Educational distribution across organizational types contained both interesting and expected information. We might guess that educational institutions hire a greater number of Ph.D.s, and perhaps even that government agencies do so, too. However, would we suspect to find no masters degrees in government agencies?

Geographics

Respondents came from every inhabited continent. There were expected clusterings in the US, Europe and Australia, although this may very well be due to the snowball sample taken.

Country	Customer HQ	Delivery to
United States	148	147
Australia	27	27
United Kingdom	10	10
Sweden	8	7
Norway	8	7
Denmark	7	4
Canada	6	6
All Others	45	43

Country	Secondary Market
Internet	25
United Kingdom	19
United States	13
Germany	9
Australia	8
Canada	7
Other E.U.	14
All Others	30

The following information is shown in Google Maps:

• Respondents' countries of birth.
• Respondents' countries of residence.
• Respondents' countries of citizenship.
• Respondents' countries of organizational headquarters.

Methodologies

The choice of methodology remains a hot topic. A large majority of respondents are using multiple methodologies at a time. Similar majorities appear to tend toward the newer methodologies, such as Test Driven Development, Extreme Programming (XP) and other Agile methodologies.

Using a single methodology: 115 (25%)

Using multiple methodologies: 317 (71%)

Using no methodologies: 16 (4%)

Using modern methodologies (TDD|Agile|XP): 305 (68%)

Not using modern methodologies (TDD|Agile|XP): 127 (28%)

Methodologies	familiar	used	using
TDD	354	260	192
Agile	311	223	202
XP	363	189	96
Flowcharting	265	163	37
RAD/Spiral RAD	205	100	43
Other than listed	35	30	22
None	3	6	16
Subtotal	1501	941	586
Structured programming	251	170	46
Rational Unified Process (RUP)	224	118	24
Top-down programming	246	146	30
Waterfall	245	120	86
Agile Unified Process (AUP)	110	35	13
Structured Systems Analysis and Design Methodology (SSADM)	66	23	2
Dynamic Systems Development Method	27	8	1
Enterprise Unified Process (EUP)	30	8	1
Virtual finite state machine (VFSM)	21	8	1
Information Engineering (IE/IEM)	24	8	0
Jackson Structured Programming	33	14	0
Praxis	8	2	0
Total of bottom 12	1285	660	204
Total	2821	1631	812
OOP NB: OOP data removed - confusing the issue	399	309	79

Metrics

A substantial portion of respondents (37%) reported using no metrics whatsoever. Of those that did, the most popular metrics appear to be the simplest, especially code coverage and source lines of code.

Metrics	familiar	used	using
None			181
Code Coverage	315	219	135
Source Lines of Code	355	271	124
Coupling	244	137	67
Cohesion	201	120	64
Cyclomatic Complexity	193	128	53
Other than listed	13	13	29
Subtotal	1321	888	653
Bugs per Line of Code	227	88	45
Order of Growth (Big O)	213	129	44
Number of classes and interfaces	200	104	40
Other	11	11	23
Function Point Analysis	169	55	16
Martin Metrics	39	28	16
Number of lines of customer requirements	28	13	7
Total of remainder	876	417	168
Total	2197	1305	821

Requirements

Who gathers requirements?
Technical Staff	179	39.96%
Product Management	146	32.59%
Other	57
Customer	47
Sales Staff	15
Not Gathered	4
Total:	448

How are requirements tracked?
Document	167	37.28%
Issues Product	78	17.41%
Other	50
Not Tracked	45
Requirements Product	39
Spreadsheet	34
Custom System	30
Presentation Software	5
Total:	448

Are requirements associated with individual software features? 329 (73)%

Are requirements associated with acceptance tests? 198 (44%)

Are requirements associated with integration tests? 156 (35%)

Are requirements associated with unit tests? 161 (36%)

How are requirements documented?
requirementsDocument	288
developerDocs	193
testCode	147
userDocument	143
testDescriptions	83
storyCards	60
other	44
null	13
CRCCards	5
Total:	976

Testing

Comments: Programmers are performing more unit, integration, code coverage and database tests than they are required to. However, they are required to perform more acceptance testing than they do.

Tests	Required	Created
unit	269	358
integration	240	256
code coverage	92	111
acceptance	237	172
database	60	68
other	34	23

Environment

Eclipse topped the list of favorite development tools. It is perhaps surprising that the venerable vi took a strong second place. Java is by far the most popular language.

Development Tools
Eclipse	206
vi	179
Visual Studio	133
emacs	96
Other text editor	84
Intellij Idea	34
XCode	22
NetBeans	19
TextMate	16
Delphi	9
JBuilder	8
C++ Builder	7
Total:	813

Languages
Java	327
SQL	204
Javascript	182
Ruby	164
Bash/Shell	131
C++	128
C#	111
C	105
Perl	94
XSLT	88
Python	84
Xpath	84
PHP	68
Basic	31
Xquery	31
AspectJ	15
Objective C	14
Matlab	9
Tcl	8
Pascal	8
Ocaml	4
Lisp	4
Fortran	3
Cobol	3
Prolog	3
Ada	2
Other Aspect	2
Haskell	2
Lua	2
Rebol	2
Smalltalk	2
Peoplesoft	2
M204 User Language	2
IDL	2
RPG	2
AspectC++	1
In-house	1
WSDL	1
Octave	1
Eiffel	1
Scheme	1
Erlang	1
Forth	1
SQR	1
Forte	1
Siebel	1
AWK	1
SAS	1
JCL	1
REXX	1
M4	1
Scheme	1
X++	1
Visual Objects	1
SPARQL	1
Lotus Notes	1
R	1
PXSL	1
Assembler	1
Visual Foxpro	1
ColdFusion	1
2E	1
webMethods Flow	1
D	0
Total:	1951

Revision Control
Subversion	241
CVS	178
Other	93
SourceSafe	66
ClearCase	30
PVCS	10
SCCS	7
None	7
Bonsai	1
SCM	0
Total:	633

Number of respondents running a continuous build system: 226 (50%)

Build System
ant	210
other	144
make	118
none	66
maven	61
none	52
null	14

A Selection of Comments

The following is a selection of free-form comments left by respondents. They provide an interesting look into the daily concerns of software engineers:

• "This seems to be primarily focused on developers who create packaged software, but these days most developers are working on internet sites and services."
• "We're a software driven company run by older programmers. the culture is of the old hacker type, rather than the new hacker type. they're fixed in anti-methods and slow to adopt anything new."
• "We're still changing our methodolgy. For one thing, we'd like to move towards something like Maven, or some other automated nightly build process. We just haven't had the time to do so."
• "I have no idea whether we are ISO 9000/9001 certified."
• "We've got a long way to go before I'm happy with how we do things here where I work!"