Global and Latin American female participation in evidence-based software engineering: a systematic mapping study

Abstract

Context

While the digital economy requires a new generation of technology for scientists and practitioners, the software engineering (SE) field faces a gender crisis. SE research is a global enterprise that requires the participation of both genders for the advancement of science and evidence-based practice. However, women across the world tend to be significantly underrepresented in such research, receiving less funding and less participation, frequently, than men as authors in research publications. Data about this phenomenon is still sparse and incomplete; particularly in evidence-based software engineering (EBSE), there are no studies that analyze the participation of women in this research area.

Objective

The objective of this work is to present the results of a systematic mapping study (SM) conducted to collect and evaluate evidence on female researchers who have contributed to the area of EBSE.

Method

Our SM was performed by manually searching studies in the major conferences and journals of EBSE. We identified 981 studies and 183 were authored/co-authored by women and, therefore, included.

Results

Contributions from women in secondary studies have globally increased over the years, but it is still concentrated in European countries. Additionally, collaboration among research groups is still fragile, based on a few women as a bridge. Latin American researchers contribute a great deal to the field, despite they do not collaborate as much within their region.

Conclusions

The findings from this study are expected to be aggregated to the existing knowledge with respect to women’s contribution to the EBSE area. We expect that our results bring up a reflection on the gender issue and motivate actions and policies to attract female researchers to this area.

Introduction

The current industry demands software developers and computer scientists far outpace the supply of computer science graduates. The women comprised 40% or more of the workforce in many countries¹; however, in 2014, only 17% of undergraduates in computer sciences were women. In 2016, 26% of the computing workforce were women². Today, only 12% of engineers are female³. Women occupy exclusively 17% of Information Technology jobs in the UK⁴. In Google⁵, only 19% of all Information Technology jobs and 24% of the top positions are occupied by women. Their representativeness, in universities, research, and higher Information Technology positions is notoriously lower when compared to the male gender.

Software engineering (SE) is one of the key fields involved in building digital infrastructures and applications for the Digital Economy [1]. SE problems are increasingly complex from both practical and theoretical standpoints, involving interactions between technical, behavioral, and social forces [2–4]. Despite SE discipline has progressed significantly over the past decades, it is still a considerable challenge to synthesize, understand, establish, and embody research principles to produce usable knowledge [5, 6]. Therefore, there is a need to move the field into evidence-based practices, to encourage reproducibility, and to rely less on anecdotal practice [7–9].

Evidence-based software engineering (EBSE) “provides how current best evidence from research can be integrated with practical experience and human values in the decision making process regarding the development and maintenance of software” [10]. SE, and consequently all stakeholders involved in software development, would surely benefit in adopting the evidence approach, increasing the capacity to deal with specific problems that arise from the nature of SE [10]. Therefore, EBSE has not only a strong role in the advancement of science but also in innovation and development.

A gender-diverse team is more likely to develop software that meets user’s requirements since teams formed only by men tend to make decisions considering men’s experiences, resulting in a male-slanted bias⁶. Moreover, comparing to homogeneous teams, a gender-diverse team is more productive, more creative, and more able to stay on schedule and within budget [11]. Therefore, SE research is a global enterprise requiring the participation of both genders for generalizable knowledge, the advancement of science, and EBSE [12]. However, women across the world tend to be significantly underrepresented in research both as researchers and research participants, receive less research funding, and appear less frequently than men as authors in research publications [13, 14]. The academia is predominantly male, especially in engineering areas [13, 15], and female researchers have reported that male counterparts spend more time in research related tasks, while they spend significantly more time in teaching and other academic duties [16]. However, the participation of women in research is necessary and acknowledged as a means to reduce the current bias that most experimental evidence is obtained from studies with men [12, 14] and by men [14] to inform related research field and related policy interventions.

Several initiatives, such as WIT – Women in Technology, Girls Who Code⁷, LinuxChix⁸, Girl Geek Dinners⁹, have been made to encourage, teach, and support girls to build systems and think computationally. In Brazil, Ciência sem Fronteiras (Brazil Scientific Mobility Program) was a government initiative that sent 75,000 Brazilian students abroad for STEM (Science, Technology, Engineering, and Math) training and education. This is one of the reasons that Brazil was benefited with a first rank in STEM gender equality¹⁰. There are other movements in Brazil, e.g., SBC Digital Girls¹¹ and PyLadies¹², focusing on increasing women’s participation in technology and computing.

Understanding female participation and collaboration on EBSE is particularly important not only to investigate empirical software engineering research bias but also to inform policy interventions. Collaboration is critical to the scientific enterprise and knowledge of collaboration patterns is useful to develop strategies that address the gender imbalance in science [17, 18]. Currently, we can highlight B. Kitchenham as one of the pioneering women in EBSE. E. Mendes, N. Juristo, O. Brereton, and S. Fabbri are women who also stand out as researchers in EBSE. Despite the aforementioned names, in EBSE, as far as we know, there are no studies that analyze who are the women working on EBSE and how they collaborate.

This work aims to identify the state of the art on women who have contributed to consolidate EBSE. By conducting a systematic mapping study (SM), we searched studies published until May 2019 and investigated the following aspects: (i) who are the women that research on secondary studies; what are the main topics covered by their studies; how these women are geographically distributed; their contribution over the years; and ii) their collaboration relationships, authorships, and co-authorships. Our goal is to explore both global and Latin American relationships.

This study intends to highlight women who work in EBSE and encourage other women researchers to join them. As the main results, we observed that women, especially European researchers, have contributed to the advancement of the EBSE area; however, collaboration among research groups is small. Therefore, it is important to develop the mentality of collaboration, which is necessary for the building of a mature and innovative research area.

The remainder of this work is organized as follows. The “Women representation in STEM” section presents related work on how women are represented in STEM. The “Research method” section discusses the research method applied to perform our SM. The “Results” section discusses the results, their implications, and limitations. Finally, the “Conclusions” section concludes this work and presents directions for future work.

Women representation in STEM

Ada Byron was a pioneer woman in computing since the area has emerged nearly two centuries ago, being the first programmer in history. Other women also participated in the evolution of computing, such as Grace Hopper who designed COBOL language, Katherine Johnson who worked on calculations for Project Apollo, and Margaret Hamilton who coined the software engineering term. Throughout the twentieth century, programming was predominantly done by women; however, this number steadily dropped with the advent of the home computer, and the importance of women decreased. The gender disparity and the lack of women in computing were first noticed by Anita Borg, a strong advocate of women in technology. At the beginning of the twenty-first century, only 28% of women were studying computer science at the post-secondary level [19], and several attempts were made to encourage more women to work in STEM.

Considering the participation of women in a scientific career, Gallivan and Benbunan-Fich [20] found that women are underrepresented among Information Systems (IS) faculty with doctoral degrees and only two women were within the top 30 researchers in their specific scientific areas. They analyzed 12 of the most important SE journals for 2 years to define the ratio of women to men as authors as well as members of editorial boards and editors in chief. They report that with respect to authorship there is a strong bias in favor of men. Regarding editorial boards, the difference was even higher: 90.5% of editors in chief are men, 76.1% of associate editors, and 82.1% of editorial board members are also men.

Unlike Gallivan and Benbunan-Fich [20], our work does not compare productivity regardless of gender. Our focus is to identify women who contribute to the improvement of EBSE, especially in secondary studies, their connection networks, and what are the most trending topics for the EBSE female community.

Research method

An SM begins with a planning phase, which includes the formulation of Research Question (RQs) and definition of inclusion and exclusion criteria, followed by search and screening of relevant studies. The data extraction activity for an SM is broad and the analysis of mapping does not include the use of in-depth analysis techniques such as meta-analysis, but rather totals and summaries.

This SM was conducted to identify women’s contribution to EBSE. Based on this goal, two RQs were created. Table 1 presents them, as well as the rationale for considering them.

Table 1 RQs and their rationales

The strategy used to identify potentially relevant studies to answer our RQs is described as follows.

We went through a manual search to identify the secondary studies conducted and also guidelines, processes, and experiences describing how to conduct or improve conducting secondary studies in SE. First of all, we defined the source venues (journals and proceedings) recognized and more targeted to EBSE, as follows [21]:

• ESEM - International Symposium on Empirical Software Engineering and Measurement;

• EASE - International Conference on Evaluation and Assessment in Software Engineering;

• EMSE - Empirical Software Engineering Journal;

• IST - Information and Software Technology Journal;

• TSE - IEEE Transactions on Software Engineering;

• IEEE-SW - IEEE Software; and

• JSS - Journal of Systems and Software.

After defining the sources, the process of identifying relevant literature started. As illustrated in Fig. 1, a total of 981 studies, which were published until May 2019, were returned.

Fig. 1

Phases of the study selection activity

Studies were included in the SM if they met the following inclusion criteria (IC):

• ∙ IC1: The study addresses secondary study; AND

• ∙ IC2: The study is authored by at least one woman.

With respect to the exclusion criteria (EC), studies were excluded if:

• ∙ EC1: The study does not address secondary study;

• ∙ EC2: The study is exclusively authored by men;

• ∙ EC3: The study does not have an abstract;

• ∙ EC4: The study is just published as an abstract;

• ∙ EC5: The study is not written in English;

• ∙ EC6: The study is an older version of other study already considered;

• ∙ EC7: The study is a non-scientific study, such as editorials, summaries of keynotes, workshops, and tutorials.

The selection activity was conducted in two phases. During phase 1, two researchers manually performed the classification of authors’ gender. Gender is a prerequisite for identifying studies that fulfill the IC2. The gender identification was processed for each author with tool support, the GenderChecker¹³, which is the world’s largest database of names. For names classified as unisex, we checked personal author’s pages to find a picture. The validity of this selection process depends on the reliability of the gender classification. Given an initial set of 50 studies, we manually classified 426 authors and compared the results against the gender that was automatically assigned by the tool.

In particular, the following metrics (see Table 2) were used to determine the quality of the classification made by this tool. The tool correctly classified 421 names (FC + MC = 1.8% + 96.9% = 98.7%), 4 names were incorrectly classified (FI + MI = 0.2% + 0.7%) and 1 name (UX = 0.2%) was unclassified.

Table 2 Classification indexes

In phase 2, the second author read in detail the full text of each primary study to decide whether to include or exclude the study. The same studies were analyzed by another researcher, who independently read the studies, and disagreements (2 studies) were resolved by discussion and consensus, resulting in 798 studies being rejected. Thus, we identified 183 relevant studies from the seven sources that we searched for. The studies included in the final selection correspond to the relevant studies that met the RQs addressed by this SM. Finally, we extracted data from all 183 studies.

For the data extraction activity, the second author was responsible for extracting the data and completing the associated forms. For validation purposes, a sample comprising 20% of the total number of primary studies was selected randomly and had their data extracted by the first researcher. There was a high level of agreement (i.e., 100%) between the second author and another.

Finally, for the data synthesis, we planned to perform using classification-schemes, tables (totals and summaries), and visual representations (graphs). Details follow.

Results

In total, we identified 703 researchers who were researching and/or conducting secondary studies in EBSE. Out of these, 518 were male (73.6%), 185 were female (26.3%), and it was not possible to classify gender for 22 (3.1%). As shown in Fig. 2, the number of men is more than twice the number of women. Based on the extraction and categorization results, it was possible to answer our two RQs as discussed below.

Fig. 2

Percentage of authors who are male/female

RQ1: Who are the women investigating secondary studies and their contributions?

With this question, we investigate which and how many publications in EBSE were authored or co-authored by women. We also investigate venues (e.g., journals, conferences, etc) where women published their research. In summary, a total of 185 women who work in EBSE were identified. Out of these, 25 (13.5%) developed research on secondary studies (that we refer herein to “state of the art”) and 160 (86.4%) conducted secondary studies.

Table 3 shows the 18 women (9.7%) who contributed with more than three publications on the state of the art of secondary studies or conducting them. The first three are O. Brereton, B. Kitchenham, and E. Mendes. In particular, O. Brereton (ID 1) has been contributing heavily to the advancement of the state of the art of EBSE – 13 publications (and 7 SLR/SM conductions). Her paper alone “Lessons from applying the systematic literature review within the software engineering domain” was cited 1203 times. Kitchenham (ID 2) stands out as the author with the largest number of publications, in particular, as the main author (11 publications). It is worth mentioning that Kitchenham and her collaborators [10] were those who suggested software engineering researchers should adopt EBSE. Her publications “Procedures for performing systematic reviews” (2004) and “Guidelines for performing systematic literature reviews in software engineering” (2007) were cited 3249 and 3413 times, respectively¹⁴. Recently, she and other researchers, including O. Brereton, released the book “Evidence-Based Software Engineering and Systematic Reviews” [22].

Table 3 Women and their contributions

Google Scholar ranks E. Mendes (ID 3) as #25 among Empirical Software Engineering Scholars. She is an editorial board member of some of the leading journals in the fields of SE, e.g., IEEE Transactions on Software Engineering. She is also a former editorial board member of the Empirical Software Engineering Journal (2007 to 2011).

The publications address issues related to 44 different SE topics. To establish these topics, we initially created a preliminary classification scheme derived from the extracted data by using topics drawn from the interpretation of the studies. We reduced/refined the topics and then we grouped the studies into these topics.

The five main topics addressed were (see Table 4): state of the art on secondary studies, software development, software process, software testing, and software requirements. The studies on the state of the art of secondary studies addressed the following subtopics: search for studies (6 studies), SLR conduction (6 studies), Visual Text Mining (5 studies), selection of studies (4 studies), SLR process (4 studies), support tool (4 studies), systematic mapping (3 studies), SLR update (3 studies), and SLR replication (1 study).

Table 4 Studies’ topic

Out of the 36 studies on the state of the art of secondary studies, 15 had a woman as the first author. Some examples, but not the only ones, are as follows: ∙ Kitchenham has led investigations on the SLR process in general [28], including the impact of limited search procedures for SLRs [26]; ∙ one of Brereton’s focus has discussed barriers faced by novices in conducting SLRs [23]; Mendes has also contributed in this perspective [45]; ∙ together, Kitchenham and Brereton have also evaluated the value of mapping studies as the basis for further research [27, 31, 33]; ∙ S. Fabbri [82] idealized and coordinated the development of a tool called StArt (State of the Art through Systematic Review)¹⁵, which helps SLR researchers during conducting SLR from protocol creation to results presented through graphics. There has been an increasing interest in the use of VTM techniques as supporting tools for SLRs [46].

To offer a general view of efforts in EBSE, a distribution of the 183 selected studies over the years is shown in Fig. 3. It presents an annual increase in the number of secondary studies conducted. As this figure suggests, there is an increase from 3 studies in 2006 to 26 in 2015, maintaining an average of 11 per year. Publications on the state of the art maintained an average of 2.6 studies per year. From 2010 to 2013, 4 studies were published per year (16 studies in total). From 2015 to 2016, 10 studies (5 per year, in median) were published. From 2014 to 2019, 124 studies were published, increasing to 20 studies published per year. In summary, publications on the state of the art totaled 21 studies, while those that describe secondary studies conduction totaled 194, nine times more than about the state of the art. Based on our results, we can observe there is a growing interest of women in participation in research on EBSE, mainly conducting secondary studies. The increase of publications made by women during the last 5 years evidence this fact.

Fig. 3

Female publication through the years

As shown in Fig. 4, in the first years (2004–2009), ESEM and EASE were the vehicles where a greater number of secondary studies were published. However, from 2010 to 2019, EASE concentrated the publications. 28 studies were published in ESEM conferences and 65 in EASE. IST is the most opted by the majority of EBSE researchers, totaling 105 publications.

Fig. 4

Publications by source

Analyzing ESEM’2019 and EASE’2019, both prime international conferences on EBSE, we respectively found out that women account for approximately 11.1% and 27.3% of their program committees. Considering IST journal, there are 7 women of a total of 30 members in the editorial board, i.e., approximately 23.3%. Moreover, we can observe that women participation as general chairs, program chairs, guest editors, and other organizational roles, is smaller than male participation. An inspiring example is E. Mendes, who was the general chair of ESEM’2017.

RQ2: How is the collaboration among women investigating secondary studies?

With this question, we investigate women with the highest number of publications in EBSE and the collaboration network among women researchers.

Figure 5 shows the geographical distribution of these women. The circle represents the presence of one or more women researching secondary studies. The color and size of the circle are proportional to the number of female researchers residing in the country. Although there is a dispersion of women researching and/or conducting secondary studies around the world, there is a concentration of female researchers in European countries as England, Spain, Germany, Sweden, Italy, Austria, France, Belgium, Denmark, Norway, Finland, Ireland, and the Netherlands. Initially, from 2004 to 2006, secondary studies were adopted mainly by female European researchers, such as B. Kitchenham, O. Brereton, and E. Mendes. In 2007, there was the participation of two Brazilians, Malheiros and Höhm, and one Venezuelan, Padua. Only in 2010, the number of female authors from non-European countries increased, reaching the current situation represented in Fig. 5.

Fig. 5

Women geographical distribution

More details on the geographical distribution of female researchers can be found in Table 5. For example, Brazil and Spain are the countries with the highest number of female researchers. In Brazil, out of 33 women, 22 have one publication; in Spain, 14 out of 21 researchers also have one publication. England has 10, but they are highly much more active in the area.

Table 5 Women and their countries

We also used graphs to answer RQ2, because they are an alternative visual representation that can be used to represent findings, showing connections among concepts and findings (e.g., authors’ collaborations). Information collected in our SM was restructured to be used in an open source tool called PEx-Graph¹⁶, which created interesting graphs, showed in Figs. 6, 7, 8, 9. In these figures, black nodes represent female authors, white nodes their respective publications, and edges represent connections between the female authors and their publications.

Fig. 6

Collaboration between women

Fig. 7

Details of collaborations between women—subgraph A

Fig. 8

Details of collaborations between women—subgraphs B-F

Fig. 9

Collaborations between Latin American female researchers

From the collected data, 78 female authors and 175 female co-authors were identified. Note that (the same woman can be the main author in one study and a co-author in another one). 42.6% of the included studies were first authored by a woman. In general, there is no strong collaboration among female authors (see Fig. 6). The lack of collaboration is perceived by several groups containing only one publication (white circle) and their respective female authors (black circles connected to the white one), spread around the graph. There are only six collaboration groups among women, detailed in Fig. 7 (details of subgraph A), Fig. 8 (subgraph B to F), and Fig. 9 (subgraph G).

At the top right of Fig. 7, two authors, B. Kitchenham and O. Brereton, stand out for the largest number of collaborative publications. These authors also collaborate with other authors, e.g., E. Mendes who is the one who makes the connection between European authors and authors of other nationalities, including Brazil, Malaysia, and New Zealand. In general, there is a prevalence of collaboration from the European authors among them.

In Fig. 8, it is possible to observe there is also a collaboration between European and American authors. The North American author C. Seaman (see Fig. 8e) is a link for collaboration with a Brazilian author. Similarly to C. Seaman is N. Juristo, a European author (see Fig. 8f) also collaborates with Latin American authors, as the Venezuelan A. Padua.

In Figs. 9 and 10, it is possible to observe the nature of female collaborations in Latin America. There are 37 Latin American female researchers, 33 from Brazil, 1 from Cuba, 1 from Chile, 1 from Mexico, and 1 from Venezuela. There is no collaboration among female researchers from these countries. In fact, the collaboration among female researchers only happens in Brazil and S. Fabbri is one of those who make the connections and has made contributions to the state of the art on secondary studies, according to Table 3.

Fig. 10

Latinas geographical distribution

Discussions

This section summarizes the main findings and discusses the relevance of this work to the EBSE community. In short, this SM presents an overview of women in EBSE around the world, their contribution, and the collaboration network as well. This section also discusses the main threats to the validity of our SM.

As a result of this work, it was observed that the number of women contributing to the EBSE area was still small compared to men. Different factors might explain these gender differences. As fewer women are working in the area, it is somewhat expected to find fewer contributions from them. Second, it has been reported that faculty usually prefers to collaborate with researchers of the same sex [101, 102]. As more men are working in the area, it might also explain the involvement of fewer women.

It is relevant to remind that women play a critical role in science and that their participation should be strengthened. One encouraging news is that the contribution of women in EBSE has increased over the years when compared to their past contributions to the field (see Fig. 3). This increase is mainly due to important women, such as O. Brereton and B. Kitchenham, conducting and investigating secondary studies in SE.

There are several national and regional initiatives in Brazil to attract women to technology. A positive and successful example is the Digital Girls Program developed by the Brazilian Computer Society. This program disseminates computer science to high school and elementary school students and it has already spread to several regions of Brazil, such as Amazonas¹⁷ and Rio Grande do Sul¹⁸.

While promoting women enrollment in STEM courses is important to increase women’s participation in the computing workforce, other policies are necessary to keep women in the area, including research in academia. In this perspective, following a request signed by Brazilian women scientists in 2019, the National Council for Scientific and Technological Development (CNPq) will add a field in the main platform for the inclusion of academic activities, publications, and research in the country, for scientists to inform the maternity and paternity leave periods. The information will be optional and the idea is to understand the impact of the birth and adoption of children in the career of scientists. Especially for women, the information would help explain a drop in productivity in the period of motherhood. The information can be used, for example, in the evaluation of the productivity of a researcher, an important criterion in the judgment of proposals of scholarship, since the period of maternity leave can affect directly the production of articles and other results.

Despite the increase in the number of female authors and the geographical dispersion worldwide, no great collaboration was found among them. For example, authors B. Kitchenham and O. Brereton concentrate the largest amount of publications collaborate but do not have many collaborations with other research groups. There is also a small representation from African countries.

The analysis of the Latin American contribution and collaborations generated some possible insights. First, there is a high number of women from Latin American countries contributing to EBSE, 37 in total. The North American female contribution (Canada + USA) is 18. Out of 37 Latin Americans, two are ranked as the top 20 female researchers contributing to the state of the art on secondary studies. However, there is little collaboration among Latin Americans due to different reasons. One possible explanation is the lack of networking among these researchers. Second, they might be networked, but they might face challenges as a lack of resources or incentives to prioritize such collaborations. In Brazil, there is a strong collaboration among 6 researchers.

Regarding our results with respect to Latin America, it is clear there is a need for more collaboration in this region. Few studies are discussing the lack of research collaboration in Latin America (e.g. [103]; therefore, it deserves further exploration. In the same perspective, there are several reported principles, particularly in the Health field, that enable truly cooperative research partnerships [104, 105], as mutual trust, shared decision-making, national ownership, emphasis on getting research findings into policy and practice, and the development of national research capacity. They should be monitored by funding agencies behind investment in research. Both North-South and South-South research collaboration should be encouraged.

Networking is extremely valuable for sharing information/resources, acquiring expertise, learning, debating ideas, knowledge of what others are doing, etc. Moreover, currently, collaboration is the foundation of much of STEM research. STEM research is done by teams composed of scientists from different fields. Two initiatives could be highlighted: (i) International Software Engineering Research Network (ISERN), a community that believes software engineering research needs to be performed in an experimental context. The main purpose of this network is to encourage and support the collaboration and exchange of results and personnel among researchers; and (ii) LinkedIn, a “social” network for professionals. LinkedIn has the group “Systematic Literature Review in Software Engineering (SLR in SE)”¹⁹ containing 1181 members. In this group, members publish conference calls, invitations for research questionnaire surveys, requests for reviews, etc.

Maturity of research

As an indication for the maturity of research developed by women in EBSE, we can mention vehicles where their publications appear (as previously shown in Fig. 4). A number of them were published in proceedings of conferences (EASE = 65 and ESEM = 28; 65+28 = 93 publications), but most of them are found in journals (IST = 155; ESE = 8; TSE = 3; IEEE-SW = 2; JSS = 4; 155+8+3+2+4 = 172 publications). Since journals usually prioritize articles with mature and validated results, the increase of publications in journals from 2015 to 2019 is a strong indication that research on secondary studies is becoming mature. It is worth highlighting the proportion of publications on state of the art (i.e., 21 studies) is nine times lower than publications on secondary studies conducted (194). The main focuses into state of the art are as follows: (i) the search for studies (6 studies were found) and (ii) report of experiences gained during conducting SM (5 studies); however, specific studies on SM are scarce (3 studies), requiring more effort to advance and consolidate research on the state of the art, including the SM process. In addition, the group of female authors who develop research on the state of the art is quite small (13) if compared to the number of men investigating it (185).

We also analyzed if there was a country leading investigations in secondary studies. We observed that the majority of publications were developed by women located in England. The remaining research groups are spread out, in particular, in other European countries. As illustrated in Fig. 6, we also noticed small collaboration among women from different countries. Through collaboration, it is possible that parties involved see different and complementary aspects of a problem and together search for solutions broader than the one constructed individually. In summary, research collaborations increase research productivity and quality; hence, collaboration among research groups is necessary to make progress. We encourage the increase of collaborative research also in EBSE. Moreover, a practical action to attract new female researchers to EBSE is to draw attention to this issue in main events in this area, as EASE and ESEM, and also social networks such as Linkedin.

Threats to validity

The main threats to the validity of our SM are described as follows.

• Missing relevant studies: Our SM was conducted based on a manual search process. We believe that all relevant studies were identified, although we cannot rule out that our actions could have led to important studies that could have been missed. To minimize problems in the search process, we selected publication databases and journals considered the most relevant ones for EBSE. Therefore, we believe that relevant studies on secondary studies were not omitted.

• Selection studies: Another threat to our SM refers to how the studies were selected since in some cases the author’s gender was not obvious to be identified. The selection validity was guaranteed by two independent reviewers and, also, an automatic gender classification tool was used. Furthermore, whenever there were disagreements among classifications, discussions and manual searches were conducted to sort out those issues.

• Gender and sex constructs are different: We recognize that gender is a socio-cultural process, while sex is a biological quality. It is also acknowledged that the male-female frame is imperfect [106]. Because the data extracted in our SM was limited, we could not precisely determine the authors’ gender. Hence, we adopted a strategy of classifying authors’ gender by their names using an automated tool.

Conclusions

Understanding female participation and collaboration network is important not only to portrait the current scenario but also to motivate further policy interventions. Considering the importance of EBSE in the twenty-first century, and that available information is scarce and scattered, understanding the female contribution and collaboration in EBSE becomes an important question.

Our study sheds light on women’s participation, contribution, and collaboration in EBSE. To build such a panorama, we performed an SM. Our results also showed that the number of men who contributed to EBSE was more than twice the female contribution. Although there is a dispersion of women researching and/or conducting secondary studies around the world, there is a concentration of female researchers in European countries as England, Spain, Germany, Sweden, Italy, Austria, France, Belgium, Denmark, Norway, Finland, Ireland, and the Netherlands. Finally, there is a high number of women from Latin American countries contributing to EBSE when compared to global female participation (37 out of 185). Considering the collaboration among women, our results showed there is at least a woman who is the link between two world regions, which still seems to be a fragile collaboration arrangement.

It is hoped that this study will be used as a motivator for other women to contribute to EBSE and to encourage policies that support stronger collaboration at a regional and global scale. There are many possible directions for future research based on our results. Identifying the contextual factors that currently enable or hamper collaboration among female researchers is the key to better explaining our results. There is a clear lack of collaboration among Latin American researchers that needs to be further explored.

Finally, we also look for periodically updating this work to monitor the evolution of women’s participation in the EBSE. We hope the systematic methodology that we detailed herein can be replicated to other domains, contributing to the creation of a comprehensive panorama about women representativeness in STEM.