Conceptual Modeling for software engineers

During conceptual modeling, participants create a model of some aspect of their
work – the intent is to bring to light their mental models. In its simplest form, participants
draw a diagram of some aspect of their work. For instance, software engineers
may be asked to draw a data flow diagram, a control flow diagram or a
package diagram showing the important architectural clusters of their system. As
an orthogonal usage, software engineers may be asked to draw a physical map of
their environment, pointing out who they talk to and how often.
Advantages: Conceptual models provide an accurate portrayal of the user’s conception
of his or her mental model of the system. Such models are easy to collect and
require only low-tech aids (pen and paper).
Disadvantages: The results of conceptual modeling are frequently hard to interpret,
especially if the researcher does not have domain knowledge about the system.
Some software engineers are reluctant to draw, and the quality and level of details
in diagrams can vary significantly.
Examples: In one of our studies, we collected system maps from all members of the
researched group. Additionally, as we followed two newcomers to a system, we had
them update their original system maps on a weekly basis. We gave them a photocopy
of the previous week’s map, and asked them to either update it or draw a new
one. The newcomers almost exclusively updated the last week’s map.
In our group study, our instructions to the study participants were to “draw their
understanding of the system.” These instructions turned out to be too vague.Some participants drew data flow diagrams, some drew architectural clusters, others listed
the important data structures and variables, etc. Not surprisingly, the manager of the
group subsequently noted that the system illustrations reflected the current
problems on which the various software engineers were working.
We learned from this exercise that for conceptual modeling to be useful, it is
important to specify to the greatest extent possible the type of diagram required. It
is next to impossible to compare diagrams from different members of a group if
they are not drawing the same type of diagram. Of course, this limits researchers in
the sense that they will not be getting unbiased representations of a system.
Specifying that data-flow diagrams are required means that software engineers
must then think of their system in terms of data-flow.
In another project (Sayyad-Shirabad et al., 1997), we wanted to discover the
concepts and terminology that software engineers use to describe a software
system. We extracted a set of candidate technical terms (anything that was not a
common English word) from source code comments and documentation. Then we
designed a simple program that allowed software engineers to manipulate the concepts,
putting them into groups and organizing them into hierarchies. We presented
the combined results to the software engineers and then iteratively worked with
them to refine a conceptual hierarchy. Although there were hundreds of concepts in
the complex system, we learned that the amount of work required to organize the
concepts in this manner was not large.
Reporting guidelines: The most important thing to report for conceptual models is
the exact instructions given to the participants and a precise description of the tools
that they had available to them to model. The way the data is recorded should also
be outlined.