Skip to main content

Global and Latin American female participation in evidence-based software engineering: a systematic mapping study

Abstract

Context

While the digital economy requires a new generation of technology for scientists and practitioners, the software engineering (SE) field faces a gender crisis. SE research is a global enterprise that requires the participation of both genders for the advancement of science and evidence-based practice. However, women across the world tend to be significantly underrepresented in such research, receiving less funding and less participation, frequently, than men as authors in research publications. Data about this phenomenon is still sparse and incomplete; particularly in evidence-based software engineering (EBSE), there are no studies that analyze the participation of women in this research area.

Objective

The objective of this work is to present the results of a systematic mapping study (SM) conducted to collect and evaluate evidence on female researchers who have contributed to the area of EBSE.

Method

Our SM was performed by manually searching studies in the major conferences and journals of EBSE. We identified 981 studies and 183 were authored/co-authored by women and, therefore, included.

Results

Contributions from women in secondary studies have globally increased over the years, but it is still concentrated in European countries. Additionally, collaboration among research groups is still fragile, based on a few women as a bridge. Latin American researchers contribute a great deal to the field, despite they do not collaborate as much within their region.

Conclusions

The findings from this study are expected to be aggregated to the existing knowledge with respect to women’s contribution to the EBSE area. We expect that our results bring up a reflection on the gender issue and motivate actions and policies to attract female researchers to this area.

Introduction

The current industry demands software developers and computer scientists far outpace the supply of computer science graduates. The women comprised 40% or more of the workforce in many countriesFootnote 1; however, in 2014, only 17% of undergraduates in computer sciences were women. In 2016, 26% of the computing workforce were womenFootnote 2. Today, only 12% of engineers are femaleFootnote 3. Women occupy exclusively 17% of Information Technology jobs in the UKFootnote 4. In GoogleFootnote 5, only 19% of all Information Technology jobs and 24% of the top positions are occupied by women. Their representativeness, in universities, research, and higher Information Technology positions is notoriously lower when compared to the male gender.

Software engineering (SE) is one of the key fields involved in building digital infrastructures and applications for the Digital Economy [1]. SE problems are increasingly complex from both practical and theoretical standpoints, involving interactions between technical, behavioral, and social forces [24]. Despite SE discipline has progressed significantly over the past decades, it is still a considerable challenge to synthesize, understand, establish, and embody research principles to produce usable knowledge [5, 6]. Therefore, there is a need to move the field into evidence-based practices, to encourage reproducibility, and to rely less on anecdotal practice [79].

Evidence-based software engineering (EBSE) “provides how current best evidence from research can be integrated with practical experience and human values in the decision making process regarding the development and maintenance of software” [10]. SE, and consequently all stakeholders involved in software development, would surely benefit in adopting the evidence approach, increasing the capacity to deal with specific problems that arise from the nature of SE [10]. Therefore, EBSE has not only a strong role in the advancement of science but also in innovation and development.

A gender-diverse team is more likely to develop software that meets user’s requirements since teams formed only by men tend to make decisions considering men’s experiences, resulting in a male-slanted biasFootnote 6. Moreover, comparing to homogeneous teams, a gender-diverse team is more productive, more creative, and more able to stay on schedule and within budget [11]. Therefore, SE research is a global enterprise requiring the participation of both genders for generalizable knowledge, the advancement of science, and EBSE [12]. However, women across the world tend to be significantly underrepresented in research both as researchers and research participants, receive less research funding, and appear less frequently than men as authors in research publications [13, 14]. The academia is predominantly male, especially in engineering areas [13, 15], and female researchers have reported that male counterparts spend more time in research related tasks, while they spend significantly more time in teaching and other academic duties [16]. However, the participation of women in research is necessary and acknowledged as a means to reduce the current bias that most experimental evidence is obtained from studies with men [12, 14] and by men [14] to inform related research field and related policy interventions.

Several initiatives, such as WIT – Women in Technology, Girls Who CodeFootnote 7, LinuxChixFootnote 8, Girl Geek DinnersFootnote 9, have been made to encourage, teach, and support girls to build systems and think computationally. In Brazil, Ciência sem Fronteiras (Brazil Scientific Mobility Program) was a government initiative that sent 75,000 Brazilian students abroad for STEM (Science, Technology, Engineering, and Math) training and education. This is one of the reasons that Brazil was benefited with a first rank in STEM gender equalityFootnote 10. There are other movements in Brazil, e.g., SBC Digital GirlsFootnote 11 and PyLadiesFootnote 12, focusing on increasing women’s participation in technology and computing.

Understanding female participation and collaboration on EBSE is particularly important not only to investigate empirical software engineering research bias but also to inform policy interventions. Collaboration is critical to the scientific enterprise and knowledge of collaboration patterns is useful to develop strategies that address the gender imbalance in science [17, 18]. Currently, we can highlight B. Kitchenham as one of the pioneering women in EBSE. E. Mendes, N. Juristo, O. Brereton, and S. Fabbri are women who also stand out as researchers in EBSE. Despite the aforementioned names, in EBSE, as far as we know, there are no studies that analyze who are the women working on EBSE and how they collaborate.

This work aims to identify the state of the art on women who have contributed to consolidate EBSE. By conducting a systematic mapping study (SM), we searched studies published until May 2019 and investigated the following aspects: (i) who are the women that research on secondary studies; what are the main topics covered by their studies; how these women are geographically distributed; their contribution over the years; and ii) their collaboration relationships, authorships, and co-authorships. Our goal is to explore both global and Latin American relationships.

This study intends to highlight women who work in EBSE and encourage other women researchers to join them. As the main results, we observed that women, especially European researchers, have contributed to the advancement of the EBSE area; however, collaboration among research groups is small. Therefore, it is important to develop the mentality of collaboration, which is necessary for the building of a mature and innovative research area.

The remainder of this work is organized as follows. The “Women representation in STEM” section presents related work on how women are represented in STEM. The “Research method” section discusses the research method applied to perform our SM. The “Results” section discusses the results, their implications, and limitations. Finally, the “Conclusions” section concludes this work and presents directions for future work.

Women representation in STEM

Ada Byron was a pioneer woman in computing since the area has emerged nearly two centuries ago, being the first programmer in history. Other women also participated in the evolution of computing, such as Grace Hopper who designed COBOL language, Katherine Johnson who worked on calculations for Project Apollo, and Margaret Hamilton who coined the software engineering term. Throughout the twentieth century, programming was predominantly done by women; however, this number steadily dropped with the advent of the home computer, and the importance of women decreased. The gender disparity and the lack of women in computing were first noticed by Anita Borg, a strong advocate of women in technology. At the beginning of the twenty-first century, only 28% of women were studying computer science at the post-secondary level [19], and several attempts were made to encourage more women to work in STEM.

Considering the participation of women in a scientific career, Gallivan and Benbunan-Fich [20] found that women are underrepresented among Information Systems (IS) faculty with doctoral degrees and only two women were within the top 30 researchers in their specific scientific areas. They analyzed 12 of the most important SE journals for 2 years to define the ratio of women to men as authors as well as members of editorial boards and editors in chief. They report that with respect to authorship there is a strong bias in favor of men. Regarding editorial boards, the difference was even higher: 90.5% of editors in chief are men, 76.1% of associate editors, and 82.1% of editorial board members are also men.

Unlike Gallivan and Benbunan-Fich [20], our work does not compare productivity regardless of gender. Our focus is to identify women who contribute to the improvement of EBSE, especially in secondary studies, their connection networks, and what are the most trending topics for the EBSE female community.

Research method

An SM begins with a planning phase, which includes the formulation of Research Question (RQs) and definition of inclusion and exclusion criteria, followed by search and screening of relevant studies. The data extraction activity for an SM is broad and the analysis of mapping does not include the use of in-depth analysis techniques such as meta-analysis, but rather totals and summaries.

This SM was conducted to identify women’s contribution to EBSE. Based on this goal, two RQs were created. Table 1 presents them, as well as the rationale for considering them.

Table 1 RQs and their rationales

The strategy used to identify potentially relevant studies to answer our RQs is described as follows.

We went through a manual search to identify the secondary studies conducted and also guidelines, processes, and experiences describing how to conduct or improve conducting secondary studies in SE. First of all, we defined the source venues (journals and proceedings) recognized and more targeted to EBSE, as follows [21]:

  • ESEM - International Symposium on Empirical Software Engineering and Measurement;

  • EASE - International Conference on Evaluation and Assessment in Software Engineering;

  • EMSE - Empirical Software Engineering Journal;

  • IST - Information and Software Technology Journal;

  • TSE - IEEE Transactions on Software Engineering;

  • IEEE-SW - IEEE Software; and

  • JSS - Journal of Systems and Software.

After defining the sources, the process of identifying relevant literature started. As illustrated in Fig. 1, a total of 981 studies, which were published until May 2019, were returned.

Fig. 1
figure1

Phases of the study selection activity

Studies were included in the SM if they met the following inclusion criteria (IC):

  1. IC1: The study addresses secondary study; AND

  2. IC2: The study is authored by at least one woman.

With respect to the exclusion criteria (EC), studies were excluded if:

  1. EC1: The study does not address secondary study;

  2. EC2: The study is exclusively authored by men;

  3. EC3: The study does not have an abstract;

  4. EC4: The study is just published as an abstract;

  5. EC5: The study is not written in English;

  6. EC6: The study is an older version of other study already considered;

  7. EC7: The study is a non-scientific study, such as editorials, summaries of keynotes, workshops, and tutorials.

The selection activity was conducted in two phases. During phase 1, two researchers manually performed the classification of authors’ gender. Gender is a prerequisite for identifying studies that fulfill the IC2. The gender identification was processed for each author with tool support, the GenderCheckerFootnote 13, which is the world’s largest database of names. For names classified as unisex, we checked personal author’s pages to find a picture. The validity of this selection process depends on the reliability of the gender classification. Given an initial set of 50 studies, we manually classified 426 authors and compared the results against the gender that was automatically assigned by the tool.

In particular, the following metrics (see Table 2) were used to determine the quality of the classification made by this tool. The tool correctly classified 421 names (FC + MC = 1.8% + 96.9% = 98.7%), 4 names were incorrectly classified (FI + MI = 0.2% + 0.7%) and 1 name (UX = 0.2%) was unclassified.

Table 2 Classification indexes

In phase 2, the second author read in detail the full text of each primary study to decide whether to include or exclude the study. The same studies were analyzed by another researcher, who independently read the studies, and disagreements (2 studies) were resolved by discussion and consensus, resulting in 798 studies being rejected. Thus, we identified 183 relevant studies from the seven sources that we searched for. The studies included in the final selection correspond to the relevant studies that met the RQs addressed by this SM. Finally, we extracted data from all 183 studies.

For the data extraction activity, the second author was responsible for extracting the data and completing the associated forms. For validation purposes, a sample comprising 20% of the total number of primary studies was selected randomly and had their data extracted by the first researcher. There was a high level of agreement (i.e., 100%) between the second author and another.

Finally, for the data synthesis, we planned to perform using classification-schemes, tables (totals and summaries), and visual representations (graphs). Details follow.

Results

In total, we identified 703 researchers who were researching and/or conducting secondary studies in EBSE. Out of these, 518 were male (73.6%), 185 were female (26.3%), and it was not possible to classify gender for 22 (3.1%). As shown in Fig. 2, the number of men is more than twice the number of women. Based on the extraction and categorization results, it was possible to answer our two RQs as discussed below.

Fig. 2
figure2

Percentage of authors who are male/female

RQ1: Who are the women investigating secondary studies and their contributions?

With this question, we investigate which and how many publications in EBSE were authored or co-authored by women. We also investigate venues (e.g., journals, conferences, etc) where women published their research. In summary, a total of 185 women who work in EBSE were identified. Out of these, 25 (13.5%) developed research on secondary studies (that we refer herein to “state of the art”) and 160 (86.4%) conducted secondary studies.

Table 3 shows the 18 women (9.7%) who contributed with more than three publications on the state of the art of secondary studies or conducting them. The first three are O. Brereton, B. Kitchenham, and E. Mendes. In particular, O. Brereton (ID 1) has been contributing heavily to the advancement of the state of the art of EBSE – 13 publications (and 7 SLR/SM conductions). Her paper alone “Lessons from applying the systematic literature review within the software engineering domain” was cited 1203 times. Kitchenham (ID 2) stands out as the author with the largest number of publications, in particular, as the main author (11 publications). It is worth mentioning that Kitchenham and her collaborators [10] were those who suggested software engineering researchers should adopt EBSE. Her publications “Procedures for performing systematic reviews” (2004) and “Guidelines for performing systematic literature reviews in software engineering” (2007) were cited 3249 and 3413 times, respectivelyFootnote 14. Recently, she and other researchers, including O. Brereton, released the book “Evidence-Based Software Engineering and Systematic Reviews” [22].

Table 3 Women and their contributions

Google Scholar ranks E. Mendes (ID 3) as #25 among Empirical Software Engineering Scholars. She is an editorial board member of some of the leading journals in the fields of SE, e.g., IEEE Transactions on Software Engineering. She is also a former editorial board member of the Empirical Software Engineering Journal (2007 to 2011).

The publications address issues related to 44 different SE topics. To establish these topics, we initially created a preliminary classification scheme derived from the extracted data by using topics drawn from the interpretation of the studies. We reduced/refined the topics and then we grouped the studies into these topics.

The five main topics addressed were (see Table 4): state of the art on secondary studies, software development, software process, software testing, and software requirements. The studies on the state of the art of secondary studies addressed the following subtopics: search for studies (6 studies), SLR conduction (6 studies), Visual Text Mining (5 studies), selection of studies (4 studies), SLR process (4 studies), support tool (4 studies), systematic mapping (3 studies), SLR update (3 studies), and SLR replication (1 study).

Table 4 Studies’ topic

Out of the 36 studies on the state of the art of secondary studies, 15 had a woman as the first author. Some examples, but not the only ones, are as follows: ∙ Kitchenham has led investigations on the SLR process in general [28], including the impact of limited search procedures for SLRs [26]; ∙ one of Brereton’s focus has discussed barriers faced by novices in conducting SLRs [23]; Mendes has also contributed in this perspective [45]; ∙ together, Kitchenham and Brereton have also evaluated the value of mapping studies as the basis for further research [27, 31, 33]; ∙ S. Fabbri [82] idealized and coordinated the development of a tool called StArt (State of the Art through Systematic Review)Footnote 15, which helps SLR researchers during conducting SLR from protocol creation to results presented through graphics. There has been an increasing interest in the use of VTM techniques as supporting tools for SLRs [46].

To offer a general view of efforts in EBSE, a distribution of the 183 selected studies over the years is shown in Fig. 3. It presents an annual increase in the number of secondary studies conducted. As this figure suggests, there is an increase from 3 studies in 2006 to 26 in 2015, maintaining an average of 11 per year. Publications on the state of the art maintained an average of 2.6 studies per year. From 2010 to 2013, 4 studies were published per year (16 studies in total). From 2015 to 2016, 10 studies (5 per year, in median) were published. From 2014 to 2019, 124 studies were published, increasing to 20 studies published per year. In summary, publications on the state of the art totaled 21 studies, while those that describe secondary studies conduction totaled 194, nine times more than about the state of the art. Based on our results, we can observe there is a growing interest of women in participation in research on EBSE, mainly conducting secondary studies. The increase of publications made by women during the last 5 years evidence this fact.

Fig. 3
figure3

Female publication through the years

As shown in Fig. 4, in the first years (2004–2009), ESEM and EASE were the vehicles where a greater number of secondary studies were published. However, from 2010 to 2019, EASE concentrated the publications. 28 studies were published in ESEM conferences and 65 in EASE. IST is the most opted by the majority of EBSE researchers, totaling 105 publications.

Fig. 4
figure4

Publications by source

Analyzing ESEM’2019 and EASE’2019, both prime international conferences on EBSE, we respectively found out that women account for approximately 11.1% and 27.3% of their program committees. Considering IST journal, there are 7 women of a total of 30 members in the editorial board, i.e., approximately 23.3%. Moreover, we can observe that women participation as general chairs, program chairs, guest editors, and other organizational roles, is smaller than male participation. An inspiring example is E. Mendes, who was the general chair of ESEM’2017.

RQ2: How is the collaboration among women investigating secondary studies?

With this question, we investigate women with the highest number of publications in EBSE and the collaboration network among women researchers.

Figure 5 shows the geographical distribution of these women. The circle represents the presence of one or more women researching secondary studies. The color and size of the circle are proportional to the number of female researchers residing in the country. Although there is a dispersion of women researching and/or conducting secondary studies around the world, there is a concentration of female researchers in European countries as England, Spain, Germany, Sweden, Italy, Austria, France, Belgium, Denmark, Norway, Finland, Ireland, and the Netherlands. Initially, from 2004 to 2006, secondary studies were adopted mainly by female European researchers, such as B. Kitchenham, O. Brereton, and E. Mendes. In 2007, there was the participation of two Brazilians, Malheiros and Höhm, and one Venezuelan, Padua. Only in 2010, the number of female authors from non-European countries increased, reaching the current situation represented in Fig. 5.

Fig. 5
figure5

Women geographical distribution

More details on the geographical distribution of female researchers can be found in Table 5. For example, Brazil and Spain are the countries with the highest number of female researchers. In Brazil, out of 33 women, 22 have one publication; in Spain, 14 out of 21 researchers also have one publication. England has 10, but they are highly much more active in the area.

Table 5 Women and their countries

We also used graphs to answer RQ2, because they are an alternative visual representation that can be used to represent findings, showing connections among concepts and findings (e.g., authors’ collaborations). Information collected in our SM was restructured to be used in an open source tool called PEx-GraphFootnote 16, which created interesting graphs, showed in Figs. 6, 7, 8, 9. In these figures, black nodes represent female authors, white nodes their respective publications, and edges represent connections between the female authors and their publications.

Fig. 6
figure6

Collaboration between women

Fig. 7
figure7

Details of collaborations between women—subgraph A

Fig. 8
figure8

Details of collaborations between women—subgraphs B-F

Fig. 9
figure9

Collaborations between Latin American female researchers

From the collected data, 78 female authors and 175 female co-authors were identified. Note that (the same woman can be the main author in one study and a co-author in another one). 42.6% of the included studies were first authored by a woman. In general, there is no strong collaboration among female authors (see Fig. 6). The lack of collaboration is perceived by several groups containing only one publication (white circle) and their respective female authors (black circles connected to the white one), spread around the graph. There are only six collaboration groups among women, detailed in Fig. 7 (details of subgraph A), Fig. 8 (subgraph B to F), and Fig. 9 (subgraph G).

At the top right of Fig. 7, two authors, B. Kitchenham and O. Brereton, stand out for the largest number of collaborative publications. These authors also collaborate with other authors, e.g., E. Mendes who is the one who makes the connection between European authors and authors of other nationalities, including Brazil, Malaysia, and New Zealand. In general, there is a prevalence of collaboration from the European authors among them.

In Fig. 8, it is possible to observe there is also a collaboration between European and American authors. The North American author C. Seaman (see Fig. 8e) is a link for collaboration with a Brazilian author. Similarly to C. Seaman is N. Juristo, a European author (see Fig. 8f) also collaborates with Latin American authors, as the Venezuelan A. Padua.

In Figs. 9 and 10, it is possible to observe the nature of female collaborations in Latin America. There are 37 Latin American female researchers, 33 from Brazil, 1 from Cuba, 1 from Chile, 1 from Mexico, and 1 from Venezuela. There is no collaboration among female researchers from these countries. In fact, the collaboration among female researchers only happens in Brazil and S. Fabbri is one of those who make the connections and has made contributions to the state of the art on secondary studies, according to Table 3.

Fig. 10
figure10

Latinas geographical distribution

Discussions

This section summarizes the main findings and discusses the relevance of this work to the EBSE community. In short, this SM presents an overview of women in EBSE around the world, their contribution, and the collaboration network as well. This section also discusses the main threats to the validity of our SM.

As a result of this work, it was observed that the number of women contributing to the EBSE area was still small compared to men. Different factors might explain these gender differences. As fewer women are working in the area, it is somewhat expected to find fewer contributions from them. Second, it has been reported that faculty usually prefers to collaborate with researchers of the same sex [101, 102]. As more men are working in the area, it might also explain the involvement of fewer women.

It is relevant to remind that women play a critical role in science and that their participation should be strengthened. One encouraging news is that the contribution of women in EBSE has increased over the years when compared to their past contributions to the field (see Fig. 3). This increase is mainly due to important women, such as O. Brereton and B. Kitchenham, conducting and investigating secondary studies in SE.

There are several national and regional initiatives in Brazil to attract women to technology. A positive and successful example is the Digital Girls Program developed by the Brazilian Computer Society. This program disseminates computer science to high school and elementary school students and it has already spread to several regions of Brazil, such as AmazonasFootnote 17 and Rio Grande do SulFootnote 18.

While promoting women enrollment in STEM courses is important to increase women’s participation in the computing workforce, other policies are necessary to keep women in the area, including research in academia. In this perspective, following a request signed by Brazilian women scientists in 2019, the National Council for Scientific and Technological Development (CNPq) will add a field in the main platform for the inclusion of academic activities, publications, and research in the country, for scientists to inform the maternity and paternity leave periods. The information will be optional and the idea is to understand the impact of the birth and adoption of children in the career of scientists. Especially for women, the information would help explain a drop in productivity in the period of motherhood. The information can be used, for example, in the evaluation of the productivity of a researcher, an important criterion in the judgment of proposals of scholarship, since the period of maternity leave can affect directly the production of articles and other results.

Despite the increase in the number of female authors and the geographical dispersion worldwide, no great collaboration was found among them. For example, authors B. Kitchenham and O. Brereton concentrate the largest amount of publications collaborate but do not have many collaborations with other research groups. There is also a small representation from African countries.

The analysis of the Latin American contribution and collaborations generated some possible insights. First, there is a high number of women from Latin American countries contributing to EBSE, 37 in total. The North American female contribution (Canada + USA) is 18. Out of 37 Latin Americans, two are ranked as the top 20 female researchers contributing to the state of the art on secondary studies. However, there is little collaboration among Latin Americans due to different reasons. One possible explanation is the lack of networking among these researchers. Second, they might be networked, but they might face challenges as a lack of resources or incentives to prioritize such collaborations. In Brazil, there is a strong collaboration among 6 researchers.

Regarding our results with respect to Latin America, it is clear there is a need for more collaboration in this region. Few studies are discussing the lack of research collaboration in Latin America (e.g. [103]; therefore, it deserves further exploration. In the same perspective, there are several reported principles, particularly in the Health field, that enable truly cooperative research partnerships [104, 105], as mutual trust, shared decision-making, national ownership, emphasis on getting research findings into policy and practice, and the development of national research capacity. They should be monitored by funding agencies behind investment in research. Both North-South and South-South research collaboration should be encouraged.

Networking is extremely valuable for sharing information/resources, acquiring expertise, learning, debating ideas, knowledge of what others are doing, etc. Moreover, currently, collaboration is the foundation of much of STEM research. STEM research is done by teams composed of scientists from different fields. Two initiatives could be highlighted: (i) International Software Engineering Research Network (ISERN), a community that believes software engineering research needs to be performed in an experimental context. The main purpose of this network is to encourage and support the collaboration and exchange of results and personnel among researchers; and (ii) LinkedIn, a “social” network for professionals. LinkedIn has the group “Systematic Literature Review in Software Engineering (SLR in SE)”Footnote 19 containing 1181 members. In this group, members publish conference calls, invitations for research questionnaire surveys, requests for reviews, etc.

Maturity of research

As an indication for the maturity of research developed by women in EBSE, we can mention vehicles where their publications appear (as previously shown in Fig. 4). A number of them were published in proceedings of conferences (EASE = 65 and ESEM = 28; 65+28 = 93 publications), but most of them are found in journals (IST = 155; ESE = 8; TSE = 3; IEEE-SW = 2; JSS = 4; 155+8+3+2+4 = 172 publications). Since journals usually prioritize articles with mature and validated results, the increase of publications in journals from 2015 to 2019 is a strong indication that research on secondary studies is becoming mature. It is worth highlighting the proportion of publications on state of the art (i.e., 21 studies) is nine times lower than publications on secondary studies conducted (194). The main focuses into state of the art are as follows: (i) the search for studies (6 studies were found) and (ii) report of experiences gained during conducting SM (5 studies); however, specific studies on SM are scarce (3 studies), requiring more effort to advance and consolidate research on the state of the art, including the SM process. In addition, the group of female authors who develop research on the state of the art is quite small (13) if compared to the number of men investigating it (185).

We also analyzed if there was a country leading investigations in secondary studies. We observed that the majority of publications were developed by women located in England. The remaining research groups are spread out, in particular, in other European countries. As illustrated in Fig. 6, we also noticed small collaboration among women from different countries. Through collaboration, it is possible that parties involved see different and complementary aspects of a problem and together search for solutions broader than the one constructed individually. In summary, research collaborations increase research productivity and quality; hence, collaboration among research groups is necessary to make progress. We encourage the increase of collaborative research also in EBSE. Moreover, a practical action to attract new female researchers to EBSE is to draw attention to this issue in main events in this area, as EASE and ESEM, and also social networks such as Linkedin.

Threats to validity

The main threats to the validity of our SM are described as follows.

  • Missing relevant studies: Our SM was conducted based on a manual search process. We believe that all relevant studies were identified, although we cannot rule out that our actions could have led to important studies that could have been missed. To minimize problems in the search process, we selected publication databases and journals considered the most relevant ones for EBSE. Therefore, we believe that relevant studies on secondary studies were not omitted.

  • Selection studies: Another threat to our SM refers to how the studies were selected since in some cases the author’s gender was not obvious to be identified. The selection validity was guaranteed by two independent reviewers and, also, an automatic gender classification tool was used. Furthermore, whenever there were disagreements among classifications, discussions and manual searches were conducted to sort out those issues.

  • Gender and sex constructs are different: We recognize that gender is a socio-cultural process, while sex is a biological quality. It is also acknowledged that the male-female frame is imperfect [106]. Because the data extracted in our SM was limited, we could not precisely determine the authors’ gender. Hence, we adopted a strategy of classifying authors’ gender by their names using an automated tool.

Conclusions

Understanding female participation and collaboration network is important not only to portrait the current scenario but also to motivate further policy interventions. Considering the importance of EBSE in the twenty-first century, and that available information is scarce and scattered, understanding the female contribution and collaboration in EBSE becomes an important question.

Our study sheds light on women’s participation, contribution, and collaboration in EBSE. To build such a panorama, we performed an SM. Our results also showed that the number of men who contributed to EBSE was more than twice the female contribution. Although there is a dispersion of women researching and/or conducting secondary studies around the world, there is a concentration of female researchers in European countries as England, Spain, Germany, Sweden, Italy, Austria, France, Belgium, Denmark, Norway, Finland, Ireland, and the Netherlands. Finally, there is a high number of women from Latin American countries contributing to EBSE when compared to global female participation (37 out of 185). Considering the collaboration among women, our results showed there is at least a woman who is the link between two world regions, which still seems to be a fragile collaboration arrangement.

It is hoped that this study will be used as a motivator for other women to contribute to EBSE and to encourage policies that support stronger collaboration at a regional and global scale. There are many possible directions for future research based on our results. Identifying the contextual factors that currently enable or hamper collaboration among female researchers is the key to better explaining our results. There is a clear lack of collaboration among Latin American researchers that needs to be further explored.

Finally, we also look for periodically updating this work to monitor the evolution of women’s participation in the EBSE. We hope the systematic methodology that we detailed herein can be replicated to other domains, contributing to the creation of a comprehensive panorama about women representativeness in STEM.

Availability of data and materials

We inform that the figures, graphics, included/excluded studies, and thus all relevant raw data will be freely available to any scientist wishing to use them for non-commercial purposes.

Notes

  1. 1.

    http://www.pewresearch.org/fact-tank/2017/03/07/ in-many-countries-at-least-four-in-ten-in-the-labor-force-are-women/

  2. 2.

    https://www.ncwit.org/

  3. 3.

    http://www.computerscience.org/resources/women-in-computer-science/

  4. 4.

    https://www.ft.com/content/e2f8ad0a-bdd6-11e5-9fdb-87b8d15baec2

  5. 5.

    https://www.google.com/diversity/

  6. 6.

    https://www.aauw.org/research/solving-the-equation/

  7. 7.

    https://girlswhocode.com/about-us/

  8. 8.

    https://www.linuxchix.org/

  9. 9.

    http://girlgeekdinners.com/

  10. 10.

    https://www.elsevier.com/connect/gender-report

  11. 11.

    http://meninas.sbc.org.br/

  12. 12.

    http://vale.pyladies.com

  13. 13.

    http://genderchecker.com/#contactus

  14. 14.

    https://scholar.google.co.uk/citations?user=CQDOm2gAAAAJ&hl=en

  15. 15.

    http://lapes.dc.ufscar.br/tools/start_tool

  16. 16.

    http://vis.icmc.usp.br/vicg/software

  17. 17.

    http://meninas.sbc.org.br/index.php/portfolio/cunhanta-digital/

  18. 18.

    http://meninas.sbc.org.br/index.php/portfolio/gurias-na-computacao/

  19. 19.

    https://www.linkedin.com/groups/2707106/

References

  1. 1

    UN. Economic and Social Council (2016) Multi-stakeholder forum on science, technology and innovation for the sustainable development goals: summary by the co-chairs: note / by the President of the Economic and Social Council, 11.. New York: UN.

  2. 2

    Boehm B (2006) A view of 20th and 21st century software engineering In: 28th International Conference on Software Engineering (ICSE), 12–29.

  3. 3

    Easterbrook S (2014) From computational thinking to systems thinking: a conceptual toolkit for sustainability computing In: 2nd International Conference on Information and Communication Technologies for Sustainability, 235–244.

  4. 4

    Narayanan A, Vallor S (2014) Why software engineering courses should include ethics coverage. Commun ACM 57(3):23–25.

    Article  Google Scholar 

  5. 5

    Sjøberg DIK, Dybå T, Jørgensen M (2007) The future of empirical methods in software engineering research In: 2007 Future of Software Engineering (FOSE), 358–378.

  6. 6

    Krein JL, Knutson CD (2010) A case for replication: synthesizing research methodologies in software engineering In: 1st International Workshop on Replication in Empirical Software Engineering Research (RESER), 10.

  7. 7

    Kitchenham BA, Budgen D, Brereton OP, Turner M, Charters S, Linkman S (2007) Large-scale software engineering questions - expert opinion or empirical evidence?IET Softw 1(5):161–171.

    Article  Google Scholar 

  8. 8

    Wohlin C (2016) Is there a future for empirical software engineering? In: 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 1–111.

  9. 9

    Dybå T, Bergersen GR, Sjøberg DIK (2016) Evidence-based software engineering In: Perspectives on Data Science for Software Engineering, 149–153.. IEEE.

  10. 10

    Kitchenham BA, Dybå T, Jorgensen M (2004) Evidence-based software engineering In: 26th International Conference on Software Engineering (ICSE), 273–281.. IEEE.

  11. 11

    Barker L, Mancha C, Ashcraf C (2014) What is the impact of gender diversity on technology business performance? Research Summary, 8.. National Center for Women & Information Technology (NCWIT). https://www.ncwit.org/sites/default/files/resources/impactgenderdiversitytechbusinessperformance_print.pdf.

  12. 12

    Princewill CW, Jegede AS, Nordström K, Lanre-Abass B, Elger BS (2017) Factors affecting women’s autonomous decision making in research participation amongst Yoruba women of Western Nigeria. Dev World Bioeth 17(1):40–49.

    Article  Google Scholar 

  13. 13

    European Commission (2016) She Figures 2015, 224.. Research Innovation, 2015. https://ec.europa.eu/research/swafts/pdf/pub_gender_equality/she_figures_2015-final.pdf .

  14. 14

    Ovseiko PV, Greenhalgh T, Adam P, Grant J, Hinrichs-Krapels S, Graham KE, Valentine PA, Sued O, Boukhris OF, Al Olaqi NM, Al Rahbi IS, Dowd A-M, Bice S, Heiden TL, Fischer MD, Dopson S, Norton R, Pollitt A, Wooding S, Balling GV, Jakobsen U, Kuhlmann E, Klinge I, Pololi LH, Jagsi R, Smith HL, Etzkowitz H, Nielsen MW, Carrion C, SolansaDomènech M, Vizcaino E, Naing L, Cheok QHN, Eckelmann B, Simuyemba MC, Msiska T, Declich G, Edmunds LD, Kiparoglou V, Buchan AMJ, Williamson C, Lord GM, Channon KM, Surender R, Buchan AM (2016) A global call for action to include gender in research impact assessment. Health Res Policy Syst 14(1):50.

    Article  Google Scholar 

  15. 15

    Blickenstaff JC (2005) Women and science careers: leaky pipeline or gender filter?Gend Educ 17(4):369–386.

    Article  Google Scholar 

  16. 16

    Aldercotte A (2017) Asset 2016: experiences of gender equality in STEMM academia and their intersections with ethnicity, sexual orientation, disability and age -Summary report, 28.. Equality Change Unit. https://s3.eu-west-2.amazonaws.com/assets.creode.advancehe-document-manager/documents/advance-he/ECU_ASSET-report_summary_April-2017_1579103249.pdf.

  17. 17

    Börner K, Contractor N, Falk-Krzesinski HJ, Fiore SM, Hall KL, Keyton J, Spring B, Stokols D, Trochim W, Uzzi B (2010) A multi-level systems perspective for the science of team science. Sci Transl Med 2(49):49cm24.

    Article  Google Scholar 

  18. 18

    Zeng XH, Duch J, Sales-Pardo M, Moreira JAG, Radicchi F, Ribeiro HV, Woodruff TK, Amaral L (2016) Differences in collaboration patterns across discipline, career stage, and gender. PLoS Biol 14(11):1–10.

    Article  Google Scholar 

  19. 19

    Cheryan S, Plaut VC, Handron C, Hudson L (2013) The stereotypical computer scientist: Gendered media representations as a barrier to inclusion for women. Sex Roles J Res 69(1–2):58–71.

    Article  Google Scholar 

  20. 20

    Gallivan MJ, Benbunan-Fich R (2006) Examining the relationship between gender and the research productivity of IS faculty In: 2006 ACM SIGMIS CPR conference on computer personnel research: Forty four years of computer personnel research: achievements, challenges & the future (SIGMIS CPR’06). Association for Computing Machinery, New York, NY, USA, 103–113.. ACM.

  21. 21

    Zhang H, Babar MA (2010) On searching relevant studies in software engineering In: 14th International Conference on Evaluation and Assessment in Software Engineering (EASE), 111–120.

  22. 22

    Kitchenham BA, Budgen D, Brereton OP (2015) Evidence-based software engineering and systematic reviews. Chapman & Hall/CRC, Boca Raton, USA.

    Book  Google Scholar 

  23. 23

    Woodall P, Brereton OP (2006) Conducting a systematic literature review from the perspective of a Ph. D. student In: 10th International Conference on Evaluation and Assessment in Software Engineering (EASE’06), 130.. BCS Learning & Development Ltd. Swindon, GBR.

  24. 24

    Turner M, Kitchenham BA, Budgen D, Brereton OP (2008) Lessons learnt undertaking a large-scale systematic literature review In: 12th International Conference on Evaluation & Assessment in Software Engineering (EASE’08), 110–118.. BCS Learning & Development Ltd., Swindon, GBR.

  25. 25

    Kitchenham BA, Brereton OP, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering–a systematic literature review. Inf Softw Technol 51(1):7–15.

    Article  Google Scholar 

  26. 26

    Kitchenham BA, Brereton OP, Turner M, Niazi M, Linkman S, Pretorius R, Budgen D (2009) The impact of limited search procedures for systematic literature reviews – A participant-observer case study In: 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM’09), 336–345.. IEEE, Lake Buena Vista.

    Chapter  Google Scholar 

  27. 27

    Kitchenham BA, Budgen D, Brereton OP (2010) The value of mapping studies: a participant-observer case study In: 14th International Conference on Evaluation and Assessment in Software Engineering (EASE), 25–33.

  28. 28

    Kitchenham BA, Brereton OP, Turner M, Niazi MK, Linkman S, Pretorius R, Budgen D (2010) Refining the systematic literature review process – two participant-observer case studies. Empir Softw Eng 15(6):618–653.

    Article  Google Scholar 

  29. 29

    Turner M, Kitchenham BA, Brereton OP, Charters S, Budgen D (2010) Does the technology acceptance model predict actual use? a systematic literature review. Inf Softw Technol 52(5):463–479.

    Article  Google Scholar 

  30. 30

    Kitchenham BA, Pretorius R, Budgen D, Brereton OP, Turner M, Niazi M, Linkman S (2010) Systematic literature reviews in software engineering–a tertiary study. Inf Softw Technol 52(8):792–805.

    Article  Google Scholar 

  31. 31

    Kitchenham BA, Budgen D, Brereton OP (2011) Using mapping studies as the basis for further research–a participant-observer case study. Inf Softw Technol 53(6):638–651.

    Article  Google Scholar 

  32. 32

    Kitchenham BA, Brereton OP, Li Z, Budgen D, Burn A (2011) Repeatability of systematic literature reviews In: 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE’11), 46–55.. IEEE, Durham.

    Chapter  Google Scholar 

  33. 33

    Kitchenham BA, Brereton OP, Budgen D (2012) Mapping study completeness and reliability-a case study In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE’12), 126–135, Ciudad Real.

  34. 34

    Major L, Kyriacou T, Brereton OP (2012) Systematic literature review: teaching novices programming using robots. IET Softw 6(6):502–513.

    Article  Google Scholar 

  35. 35

    Verner JM, Brereton OP, Kitchenham BA, Turner M, Niazi M (2012) Systematic literature reviews in global software development: a tertiary study In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE’12), 2–11, Ciudad Real.

  36. 36

    Kitchenham BA, Brereton OP (2013) A systematic review of systematic review process research in software engineering. Inf Softw Technol 55(12):2049–2075.

    Article  Google Scholar 

  37. 37

    Marshall C, Brereton OP, Kitchenham BA (2015) Tools to support systematic reviews in software engineering: a cross-domain survey using semi-structured interviews In: 19th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 15). Association for Computing Machinery, New York, NY, USA, Article 26,1–6.. ACM.

  38. 38

    Olorisade BK, de Quincey E, Brereton OP, Andras P (2016) A critical analysis of studies that address the use of text mining for citation screening in systematic reviews In: 20th International Conference on Evaluation and Assessment in Software Engineering (EASE), 14.

  39. 39

    Budgen D, Brereton OP, Drummond S, Williams N (2018) Reporting systematic reviews: some lessons from a tertiary study. Inf Softw Technol 95:62–74.

    Article  Google Scholar 

  40. 40

    Budgen D, Brereton OP, Williams N, Drummond S (2018) The contribution that empirical studies performed in industry make to the findings of systematic reviews: a tertiary study. Inf Softw Technol 94:234–244.

    Article  Google Scholar 

  41. 41

    Kitchenham BA, Mendes E, Travassos GH (2006) A systematic review of cross-vs. within-company cost estimation studies In: 10th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 06), 81–90.. British Computer Society.

  42. 42

    Baldassarre MT, Caivano D, Kitchenham BA, Visaggio G (2007) Systematic review of statistical process control: an experience report In: 11th International Conference on Evaluation and Assessment in Software Engineering (EASE’07), 94–102, BCS Learning & Development Ltd., Swindon, GBR.

  43. 43

    Zhang H, Kitchenham BA, Pfahl D (2008) Software process simulation over the past decade: trends discovery from a systematic review. In: Elbaum S Muench J (eds)ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 08), 345–347.. ACM SIGSOFT & IEEE CS.

  44. 44

    Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics In: 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM), 367–377.

  45. 45

    Riaz M, Sulayman M, Salleh N, Mendes E (2010) Experiences conducting systematic reviews from novices’ perspective In: 14th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 10), 44–53, BCS Learning & Development Ltd., Swindon, GBR.

  46. 46

    Felizardo KR, Salleh N, Martins RM, Mendes E, MacDonell SG, Maldonado JC (2011) Using visual text mining to support the study selection activity in systematic literature reviews In: International Symposium on Empirical Software Engineering and Measurement (ESEM’ 11), 77–86.. IEEE, Banff.

    Google Scholar 

  47. 47

    Felizardo KR, Mendes E, Kalinowski M, Souza EF, Vijaykumar NL (2016) Using forward snowballing to update systematic reviews in software engineering In: 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 16). Association for Computing Machinery, New York, NY, USA, Article 53, 1–6.. ACM.

  48. 48

    Sulayman M, Mendes E (2011) An extended systematic review of software process improvement in small and medium web companies In: 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE’11), 134–143.. IET, Durham.

    Chapter  Google Scholar 

  49. 49

    Mendes E, Kalinowski M, Martins D, Ferrucci F, Sarro F (2014) Cross-vs. within-company cost estimation studies revisited: an extended systematic review In: 18th International Conference on Evaluation and Assessment in Software Engineering (EASE’14). Association for Computing Machinery, New York, NY, USA, Article 12, 1–10.. ACM.

  50. 50

    Soomro AB, Salleh N, Mendes E, Grundy J, Burch G, Nordin A (2016) The effect of software engineers personality traits on team climate and performance: a systematic literature review. Inf Softw Technol 73:52–65.

    Article  Google Scholar 

  51. 51

    Alves NSR, Mendes TS, de Mendonça MG, Spínola RO, Shull F, Seaman C (2016) Identification and management of technical debt: a systematic mapping study. Inf Softw Technol 70:100–121.

    Article  Google Scholar 

  52. 52

    Usman M, Britt R, Börstler J, Mendes E (2017) Taxonomies in software engineering: a systematic mapping study and a revised taxonomy development method. Inf Softw Technol 85:43–59.

    Article  Google Scholar 

  53. 53

    Molléri JS, Petersen K, Mendes E (2019) Cerse – catalog for empirical research in software engineering: a systematic mapping study. Inf Softw Technol 105:117–149.

    Article  Google Scholar 

  54. 54

    Felizardo KR, Nakagawa EY, Feitosa D, Minghim R, Maldonado JC (2010) An approach based on visual text mining to support categorization and classification in the systematic mapping In: 14th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 10), 34–43, BCS Learning & Development Ltd., Swindon, GBR.

  55. 55

    Felizardo KR, Andery GF, Paulovich FV, Minghim R, Maldonado JC (2012) A visual analysis approach to validate the selection review of primary studies in systematic reviews. Inf Softw Technol 54(10):1079–1091.

    Article  Google Scholar 

  56. 56

    Felizardo KR, Nakagawa EY, MacDonell SG, Maldonado JC (2014) A visual analysis approach to update systematic reviews In: 18th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 14). Association for Computing Machinery, New York, NY, USA, Article 4, 1–10.. ACM.

  57. 57

    Octaviano FR, Felizardo KR, Maldonado JC, Fabbri SCPF (2015) Semi-automatic selection of primary studies in systematic literature reviews: is it reasonable?Empir Softw Eng 20(6):1898–1917.

    Article  Google Scholar 

  58. 58

    Engström E, Runeson P, Skoglund M (2010) A systematic review on regression test selection techniques. Inf Softw Technol 52(1):14–30.

    Article  Google Scholar 

  59. 59

    Engström E, Runeson P (2011) Software product line testing–a systematic mapping study. Inf Softw Technol 53(1):2–13.

    Article  Google Scholar 

  60. 60

    Engström E, Skoglund M, Runeson P (2008) Empirical evaluations of regression test selection techniques: a systematic review In: 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 08), 22–31.. ACM.

  61. 61

    Mäntylä MV, Adams B, Khomh F, Engström E, Petersen K (2015) On rapid releases and software testing: a case study and a semi-systematic literature review. Empir Softw Eng 20(5):1384–1425.

    Article  Google Scholar 

  62. 62

    Bano M, Imtiaz S, Ikram N, Niazi M, Usman M (2012) Causes of requirement change-a systematic literature review In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 12), Ciudad Real, 22–31.. IET.

  63. 63

    Alvertis I, Koussouris S, Papaspyros D, Arvanitakis E, Mouzakitis S, Franken S, Kolvenbach S, Prinz W (2016) User involvement in software development processes. Proc Comput Sci 97:73–83. https://www.sciencedirect.com/science/article/pii/S1877050916320981.

    Article  Google Scholar 

  64. 64

    Imtiaz S, Bano M, Ikram N, Niazi M (2013) A tertiary study: experiences of conducting systematic literature reviews in software engineering In: 17th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 13). Association for Computing Machinery, New York, NY, USA, 177–182.. ACM.

  65. 65

    Bano M, Zowghi D (2015) A systematic review on the relationship between user involvement and system success. Inf Softw Technol 58:148–169.

    Article  Google Scholar 

  66. 66

    Pretorius R, Budgen D (2008) A mapping study on empirical evidence related to the models and forms used in the UML In: 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ O8). Association for Computing Machinery, New York, NY, USA, 342–344.. ACM.

  67. 67

    Mitchell SM, Seaman CB (2009) A comparison of software cost, duration, and quality for waterfall vs. iterative and incremental development: a systematic review In: 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM’ 09), Lake Buena Vista, FL, USA, 511–515.. IEEE Computer Society.

  68. 68

    Carroll C, Falessi D, Forney V, Frances A, Izurieta C, Seaman C (2015) A mapping study of software causal factors for improving maintenance In: ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 15), Beijing, China, 1–4.. IEEE.

  69. 69

    Penzenstadler B, Bauer V, Calero C, Franch X (2012) Sustainability in software engineering: a systematic literature review In: 16th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 12), Ciudad Real, 32–41.

  70. 70

    Febrero F, Calero C, Moraga MA (2014) A systematic mapping study of software reliability modeling. Inf Softw Technol 56(8):839–849.

    Article  Google Scholar 

  71. 71

    Penzenstadler B, Raturi A, Richardson D, Calero C, Femmer H, Franch X (2014) Systematic mapping study on software engineering for sustainability (SE4S) In: 18th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 14). Association for Computing Machinery, New York, NY, USA, Article 14, 1–4.. ACM.

  72. 72

    Mehmood A, Jawawi DNA (2013) Aspect-oriented model-driven code generation: a systematic mapping study. Inf Softw Technol 55(2):395–411.

    Article  Google Scholar 

  73. 73

    Younas M, Jawawi NAD, Ghani I, Fries T, Kazmi R (2018) Agile development in the cloud computing environment: a systematic review. Inf Softw Technol 103:142–158.

    Article  Google Scholar 

  74. 74

    Khatibsyarbini M, Adham MI, Jawawi NAD, Tumeng R (2018) Test case prioritization approaches in regression testing: a systematic literature review. Inf Softw Technol 93:74–93.

    Article  Google Scholar 

  75. 75

    Fernández-Sáez AM, Genero M, Chaudron MRV (2013) Empirical studies concerning the maintenance of UML diagrams and their use in the maintenance of code: a systematic mapping study. Inf Softw Technol 55(7):1119–1142.

    Article  Google Scholar 

  76. 76

    Torre D, Labiche Y, Genero M (2014) UML consistency rules: a systematic mapping study In: 18th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 14). Association for Computing Machinery, New York, NY, USA, Article 6, 1–10.. ACM.

  77. 77

    Vargas JA, García-Mundo L, Genero M, Piattini M (2014) A systematic mapping study on serious game quality In: 18th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 14). Association for Computing Machinery, New York, NY, USA, Article 15, 1–10.. ACM.

  78. 78

    Riaz M (2012) Maintainability prediction of relational database-driven applications: a systematic review In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 12), Ciudad Real, 263–272.

  79. 79

    Dieste O, Grim A, Juristo N, Saxena H (2011) Quantitative determination of the relationship between internal validity and bias in software engineering experiments: consequences for systematic literature reviews In: International Symposium on Empirical Software Engineering and Measurement (ESEM’ 11), Banff, AB, Canada, 285–294.. IEEE.

  80. 80

    Acuña ST, Castro JW, Dieste O, Juristo N (2012) A systematic mapping study on the open source software development process In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 12), Ciudad Real, 42–46.. IET.

  81. 81

    González JE, Juristo N, Vegas S (2014) A systematic mapping study on testing technique experiments: has the situation changed since 2000? In: 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 14). Association for Computing Machinery, New York, NY, USA, Article 3, 1–4.. ACM.

  82. 82

    Fabbri S, Silva C, Hernandes E, Octaviano F, Di Thommazo A, Belgamo A (2016) Improvements in the start tool to better support the systematic review process In: 20th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 16). Association for Computing Machinery, New York, NY, USA, Article 21, 1–5.. ACM.

  83. 83

    Octaviano F, Silva C, Fabbri SCPF (2016) Using the SCAS strategy to perform the initial selection of studies in systematic reviews: an experimental study In: 20th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 16). Association for Computing Machinery, New York, NY, USA, Article 25, 1–10.. ACM.

  84. 84

    Liu D, Wang Q, Xiao J (2009) The role of software process simulation modeling in software risk management: a systematic review In: 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM’ 09), Lake Buena Vista, FL, USA, 302–311.. IEEE.

  85. 85

    Sun Y, Yang Y, Zhang H, Zhang W, Wang Q (2012) Towards evidence-based ontology for supporting systematic literature review In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 12), Ciudad Real, 171–175.. IET.

  86. 86

    Li J, Zhang H, Zhu L, Jeffery R, Wang Q, Li M (2012) Preliminary results of a systematic review on requirements evolution In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE), 12–21.

  87. 87

    Assunção WKG, Lopez-Herrejon RE, Linsbauer L, Vergilio SR, Egyed A (2017) Reengineering legacy applications into software product lines: a systematic mapping. Empir Softw Eng 22(6):2972–3016.

    Article  Google Scholar 

  88. 88

    Ferreira TN, Vergilio SR, de Souza JT (2017) Incorporating user preferences in search-based software engineering: A systematic mapping study. Inf Softw Technol 90:55–69.

    Article  Google Scholar 

  89. 89

    Mariani T, Vergilio SR (2017) A systematic review on search-based refactoring. Inf Softw Technol 83:14–34.

    Article  Google Scholar 

  90. 90

    Dybå T, Kampenes VB, Sjøberg DI (2006) A systematic review of statistical power in software engineering experiments. Inf Softw Technol 48(8):745–755.

    Article  Google Scholar 

  91. 91

    Kampenes VB, Dybå T, Hannay JE, Sjøberg DI (2007) A systematic review of effect size in software engineering experiments. Inf Softw Technol 49(11):1073–1086.

    Article  Google Scholar 

  92. 92

    Kampenes VB, Dybå T, Hannay JE, Sjøberg DI (2009) A systematic review of quasi-experiments in software engineering. Inf Softw Technol 51(1):71–82.

    Article  Google Scholar 

  93. 93

    Dieste O, Padua AG (2007) Developing search strategies for detecting relevant experiments for systematic reviews In: 1st International Symposium on Empirical Software Engineering and Measurement (ESEM’ 07), Madrid, Spain, 215–224.. IEEE.

  94. 94

    Zhang H, Babar MA, Bai X, Li J, Huang L (2011) An empirical assessment of a systematic search process for systematic reviews In: 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE’ 11), Durham, 56–65.. IET.

  95. 95

    Jalali S, Wohlin C (2012) Systematic literature studies: database searches vs. backward snowballing In: 6th ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 12), Lund, Sweden, 29–38.. ACM.

  96. 96

    Badampudi D, Wohlin C, Petersen K (2015) Experiences from using snowballing and database searches in systematic literature studies In: 19th International Conference on Evaluation and Assessment in Software Engineering (EASE’ 15). Association for Computing Machinery, New York, NY, USA, Article 17, 1–10.. ACM.

  97. 97

    Oates BJ, Capper G (2009) Using systematic reviews and evidence-based software engineering with masters students In: 13th International Conference on Evaluation and Assessment in Software Engineering (EASE), 20–21.

  98. 98

    Malheiros V, Höhn E, Pinho R, Mendonça M, Maldonado JC (2007) A visual text mining approach for systematic reviews In: 1st International Symposium on Empirical Software Engineering and Measurement (ESEM’ 07), Madrid, Spain, 245–254.. IEEE.

  99. 99

    Carver JC, Hassler E, Hernandes E, Kraft NA (2013) Identifying barriers to the systematic literature review process In: ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’ 13), Baltimore, MD, USA, 203–212.. IEEE.

  100. 100

    Molléri JS, Benitti FBV (2015) Sesra: a web-based automated tool to support the systematic literature review process In: 19th International Conference on Evaluation & Assessment in Software Engineering (EASE’ 15). Association for Computing Machinery, New York, NY, USA, Article 24, 1–6.. ACM.

  101. 101

    McNeely CL, Schintler L (2010) Gender issues in scientific collaboration and workforce development: implications for a federal policy research agenda In: Workshop on the Science of Science Measurement, U.S. Office of Science and Technology Policy, Washington, DC, 15. https://www.nsf.gov/sbe/sosp/workforce/mcneely.pdf.

  102. 102

    Ceci SJ, Ginther DK, Kahn S, Williams WM (2014) Women in academic science: a changing landscape. Psychol Sci Pub Interes 15(3):75–141.

    Article  Google Scholar 

  103. 103

    Mercado-Martinez FJ (2013) Collaboration in qualitative health research in Latin America: another utopia?Texto & Contexto – Enfermagem 22:871–872.

    Article  Google Scholar 

  104. 104

    Costello A, Zumla A (2000) Moving to research partnerships in developing countries. BMJ 321(7264):827–829.

    Article  Google Scholar 

  105. 105

    Binka F (2005) North-south research collaborations: a move towards a true partnership?Trop Med Int Health 10(3):207–209.

    Article  Google Scholar 

  106. 106

    Ashcraft C, McLain B, Eger E (2016) Women in tech: the facts (2016 update). Technical report, 76.. National Center for Women & Information Technology (NCWIT). https://www.ncwit.org/sites/default/files/resources/ncwit_women-in-it_2016-full-report_final-web06012016.pdf.

Download references

Acknowledgements

The authors thank the financial support received from Brazilian funding agencies FAPESP (Grants: 2014/02244-7, 2017/06195-9) and CNPq (Grants: 401033/2016-3, 432247/2018-1).

Funding

Not applicable.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Katia Romero Felizardo.

Ethics declarations

Competing interests

The authors declare that they have no competing interests. All of us have approved the manuscript for submission and we also assure, to the best of our knowledge, that the content of the manuscript has not been published or submitted for publication elsewhere.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Felizardo, K.R., Ramos, A.M., O. Melo, C.d. et al. Global and Latin American female participation in evidence-based software engineering: a systematic mapping study. J Braz Comput Soc 27, 6 (2021). https://doi.org/10.1186/s13173-021-00109-7

Download citation

Keywords

  • Secondary study
  • Systematic mapping
  • Women in computing