Demonstrating public value to funders and other stakeholders—the journey of ELIXIR, a virtual and distributed research infrastructure for life science data

Open Science is a founding principle of ELIXIR, a pan-European research infrastructure for life science data, with 21 Member countries plus the European Molecular Biology Laboratory. The mission of ELIXIR is to coordinate bioinformatics resources so that they form a single, integrated and pan-European infrastructure, which can be used freely by academic and private-sector researchers across the globe. As a recipient of public and charitable funding, ELIXIR must demonstrate its value, and the need to produce evidence in support of this is intensifying. Our practice-led journey towards demonstrating public value is articulated around five main challenges and


INTRODUCTION
Life science research has become data-intensive, as demonstrated by the rise of bioinformatics, or 'computational biology'.Traditional 'wet labs' increasingly make space for "dry (computer) labs", and bioinformatics applications of economic and societal benefit are numerous.In the healthcare sector, for example, these applications are well recognized, and range from disease diagnostic (e.g.rare diseases) and prevention (e.g.certain cancers), to epidemics monitoring, personalized medicine (or precision medicine), and the repurposing or development of drugs, vaccines and treatments.Successful applications of bioinformatics to food production are found in farming and agriculture (e.g.crop and breeds development, pest and pathogen control), and are key to ensuring security of food supplies globally.The environment is probably the area where bioinformatics applications have the greatest unrealised potential to date.Bioinformatics can, for instance, provide solutions for pollution control and remediation, and is revolutionizing our biogeographic knowledge of species and habitats, having already provided hope for the development of new antibiotics in the face of rising bacterial resistance.
For these "big data" applications to happen, it is essential that data and related resources such as tools, cloud computing, training, and standards, are kept as openly accessible and free as possible.In this context, 2020 saw the drafting of a future UNESCO Recommendation on Open Science (UNESCO, 2020), which hence acknowledges a movement that had emerged from the scientific community and spread across sectors, including the private sector, the policy sphere and citizens.This future, international standard-setting instrument on Open Science builds on the Universal Declaration of Human Rights (1948) which specifically mentions the societal benefits that are to be derived from science: "everyone has the right freely to participate in the cultural life of the community, to enjoy the arts and to share in 1 scientific advancement and its benefits" (art.27).Open Science is also a founding principle of ELIXIR, a pan-European research infrastructure for life science data, with 21 Member countries plus the European Molecular Biology Laboratory.ELIXIR became operational in 2014, hence it is relatively young, yet some of its national-level "Nodes" are well-established, mature, and have operated for decades.The mission of ELIXIR is to coordinate bioinformatics resources so that they form a single, integrated and pan-European infrastructure, which can be used freely and typically without registration by academic and private-sector researchers across the globe.
As a recipient of public and charitable funding, ELIXIR, like other research infrastructures, must demonstrate its value.Since its inception, ELIXIR has received accolades from international stakeholders.The European Strategy Forum on Research Infrastructures (ESFRI) identified it as one of three "priority projects for implementation", in that it was "pushing the boundaries of scientific excellence, and [was] strategically relevant for Europe" (Council of the European Union, 2014).ELIXIR was also named a "Global Research Infrastructure" by the Group of Senior Officials on Global Research Infrastructures (GSOGRI, 2015), as well as assessed by an independent high-level expert group to be "performing well" and "noted for its advanced science 'services'2 and clustering activities" (European Commission, 2020).Yet, there is a growing need to collate and communicate evidence of this public value in a more systematic and detailed way.The longterm sustainability, that is, continued operation and survival, of the research infrastructure and its resources depends on this (Smith et al., 2020).
Here, we provide a perspective from those inside ELIXIR, whom for the most part are scientists by training, who know the research infrastructure deeply, and who are exposed to numerous and varied requests for evidence of public value from those funding it as well as other important stakeholders.Our practice-led journey towards demonstrating public value is articulated around five main challenges and, for each, we present our pragmatic approach for tackling it, along with worked examples.Through sharing the knowledge and experience collectively acquired during this journey, our main objective is to show that, however big the challenges appear to be, a lot can be achieved together "avec les moyens du bord", namely with the means at hand, enthusiasm and creativity.We believe that the lessons learned we conclude with are sufficiently transferable and will be inspirational to other research infrastructures as they embark on their own journeys to demonstrate public value.

Identifying ELIXIR's main impact areas and the journey's "challenges"
ELIXIR's main impact areas were not arrived at following a systematic approach.Rather, they have naturally emerged as a result of reviewing existing classifications, and carrying out informal discussions and consultations across ELIXIR and beyond.Their names changed several times along the way as we tested and refined them on different audiences with ranging awareness levels of ELIXIR.The accumulation of such hands-on learning and experience in other research infrastructures is likely to promote the emergence of more structured approaches and stepwise processes for this and other steps of impact evaluation.
Similarly, the challenges used to structure our journey-based storyline are not the result of a systematic process.Rather, they illustrate the barriers that, as a community of practice, we agreed we had faced in one way or another along the way, hoping that such a structure would resonate with multiple audience types.

2.2
Capturing relationship capital using post-event perception surveys Relationship capital, or the benefits of working together and cooperation, is naturally quite an intangible impact.To try and capture this, we have worked to develop specific survey questions to collect participants' perceptions on the effect of ELIXIR-funded flagship events on the development of personal outcomes linked to relationship capital.After a few refinements based on early trials, we have settled on a set of questions that can be adjusted to each event's objectives.These questions are systematically used for the large events (100+ attendees) which we organize and fund, such as the ELIXIR All Hands, the Biohackathon Europe, and the ELIXIR SME and Innovation series.

2.3
Indicators linked to visibility of the research infrastructure with the policy sphere Since ELIXIR became operational in 2012, we have kept a record of actions taken to increase ELIXIR's visibility with, and appreciation by, funders and the policy sphere more broadly (e.g.European Union and its various bodies and agencies, intergovernmental organizations, national funding agencies, and a wide range of other research infrastructure stakeholders).Such actions are very varied in nature and include formal consultation responses, answers to information requests of many sorts, publication of position papers, organization of events targeting policy stakeholders, and provision of case studies for policy reports.
Alongside the tracking of these efforts, we have kept a record of mentions of ELIXIR, its projects, resources and key achievements in policy documents, reports, guides for grantees, funding calls, official guidelines, communications, and so on.Some of these mentions are direct outcomes of specific actions we had taken, for example provision of a case-study for a given report, yet for many instances, the mentions cannot be attributed to a specific action from our part, reflecting many years worth of interactions and activities.What started as a reactive and manual tracking exercise has moved to be more systematic through the use of the overton.iodatabase, a searchable index of policy documents, guidelines, think tank publications and working papers.Beyond more than doubling the number of sources with such mentions, the use of the overton.iodatabase has revealed blind spots in our manual tracking (e.g.national-level sources, stakeholders we had not considered), along with mentions that we had simply missed.

Challenge #1: Enhancing understanding of what research infrastructures do
With the exception of initiated and specialized circles, research infrastructures are often understood as physical constructs, typically expensive and made of cement and steel, for instance telescopes and particle accelerators.The reality is that research infrastructures span all fields of research from energy, environment, health & food, physical sciences & engineering, social & cultural innovation, to data, computing & digital research (ESFRI, 2018).Research infrastructures can be further classed as distributed (geographically) or single-sited, as well as physical or virtual.A distributed and virtual infrastructure, such as ELIXIR, usually does not have an eye catching and awe-inspiring physical structure that non-initiated can appreciate, or at least take note of.Rather, distributed and virtual infrastructures are often "people infrastructures", where a significant asset is the highly trained workforce that develops and operates the resources provided to users.In many cases, and particularly true for ELIXIR, research infrastructure operators also lead scientific research careers (funded by non-infrastructure research budgets), meaning that it is not straightforward to estimate workforce and associated payroll sizes.Altogether, it remains that it can be a challenge to explain what research infrastructures are, and one way to bypass this is to focus on what they do and the derived impacts and public value.
In the case of ELIXIR, the added-value of the research infrastructure lies in targeted actions to coordinate existing efforts, so as to create an integrated pan-European infrastructure.To demystify ELIXIR and its added-value, we have worked to better articulate, in lay English, where we put our efforts and what we fund, namely, via: • the "ELIXIR Hub budget", made of contributions from ELIXIR Member countries and managed by the "ELIXIR Hub", its coordinating Secretariat (see https://elixir-europe.org/about-us/ financial-plan), • national funding, often in the context of national-level prioritisation processes called roadmaps for research infrastructures (see https://www.esfri.eu/national-roadmaps),and • participation in competitive grants from the European Union, for instance under the Horizon program (see https://elixir-europe.org/about-us/how-funded/eu-projects).
This has translated to main categories of direct impact (Figure 1) where pretty much any activity done under the umbrella of ELIXIR can be attached to one or more of these impact areas, and related evidence looked into and collated.These impact areas have proven to be a powerful and effective framework to communicate the value of ELIXIR's work to any type of audience, within and outside of the infrastructure.
Following the principle that "the whole is greater than the sum of its parts", much of ELIXIR's coordination work is about connecting national bioinformatics infrastructures and their experts, getting them to work together, leading to what we have called relationship capital.Evidence of this effect has been collated via perception surveys (Figure 2) on participants of flagship events in which ELIXIR allocates a significant proportion of its budget, and expects a good "return-oninvestment".
Similarly, ELIXIR uses a funding instrument called "commissioned services" to get national bioinformatics infrastructures and their experts to work together on projects to implement the organization's five-year Scientific Program (ELIXIR, 2019), and hence bring the infrastructure together like glue.Much of this work takes the form of activities articulated around five technical domains of implementation or "Platforms" (databases, tools, cloud computing, training, and standards) and a growing number of use cases or "Communities" (e.g.rare diseases, plant science), and the typical impact area is that of research efficiency.There again, working relationships develop through funded projects, in which participation from several countries is a requirement (as visualized in Figure 3), and typically results in increased mutual appreciation across national jurisdictions and cultures.Much of existing relationship capital, developed over the years, was taken advantage of in 2020 as part of a coordinated international response to the COVID-19 pandemic (Blomberg & Lauer, 2020).

Challenge #2: One infrastructure, many funders with distinct views on public value
Coordinating activities are only one part of what ELIXIR does-first and foremost, the national bioinformatics infrastructures that are part of ELIXIR develop and operate hundreds of bioinformatics resources (see https://elixir-europe.org/services).These are funded through a myriad of public and charitable sources, including some already mentioned above, but also EU "structural funds" (European Structural and Investment Funds), contributions from national governments (for instance to the budget of the European Molecular Biology Laboratory), and various national-level and international trusts and foundations (Martin et al., 2019b), and in some cases the private sector.Even at the level of one resource, the funding landscape can be complex with, for instance, Europe PMC (for scientific literature) being supported by more than 30 funders (see https://europepmc.org/Funders/).Funders all have individual missions and related strategic objectives, and hence specific requirements in terms of evidence in support of these.Increasingly, ELIXIR (its coordinating secretariat and its national Nodes) is being asked to provide such evidence, through a very diverse set of questions to demonstrate public value.
More often than not, questions have been designed for use across a range of research infrastructure types and fields, sometimes for the purpose of benchmarking, meaning that their relevance and applicability to ELIXIR will vary.Yet, important decisions are made based on responses, such as country membership (new or continued) in international research infrastructures (e.g.Swedish Research Council, 2018).It is hence crucial to have at hand relevant evidence of public value that will resonate with the funder in question, even if it does not align perfectly with the question being asked.See Table 1 (Supporting Information) for a selection of questions illustrating the breadth of evidence ELIXIR needs to have at hand.
There, ELIXIR's approach has been to gradually build its understanding of funders' expectations and needs and, based on this, to proactively collate a body of evidence.Evidence can sometimes be simple and straightforward (e.g.simple lists of examples to support narratives), sometimes more complex and advanced, requiring a team effort.This is the case for our international training database relating to human capital impact (see https://training-metrics-dev.elixir-europe.org/all-reports and Gurwitz et al., 2020): this database can be interrogated to suit specific requests for information (e.g.number of trainees for specific year ranges, their gender and career stage).
Repeated requests for information, from various funders over the years, have helped ELIXIR resource providers realise the power of better communicating the public value of their resources.This has led to an increasing number of systems being put in place to estimate user demand on, and hence usefulness of, resources (e.g.user numbers and location in a given country), beyond simply communicating about what the resource does and offers to potential users (e.g.number of entries in a database or registry).

Challenge #3: Beyond funders, other stakeholders ask for evidence of public value
Research infrastructures have many stakeholders, ranging from the scientific community (the academic and private-sector users of its resources), management and staff, "initiators" (those at the origin of the infrastructure, who may or may not still work there), country hosts, funders (e.g.national funders, foundations and trusts, and in some case the private sector), and local/regional/national authorities, all the way to civil society.Each of these stakeholders will have different "interests" when it comes to demonstrating public value.Civil society, for instance, would be interested in applications of societal benefit and value for money, whilst regional authorities would focus more on innovation ecosystems, and initiators would be more concerned with arguments for fundraising (see further detail in Table 1 of OECD, 2019).Although only a portion of these stakeholders will fund the research infrastructure directly (e.g.grants from national funders, foundations and trusts, contributions paid by national authorities to be part of ELIXIR), or even indirectly (civil society via taxation), all stakeholders are influential at various scales and need to be satisfied of the public value of the infrastructure.
ESFRI is one such stakeholder which does not fund ELIXIR, yet has strong influence on how the infrastructure is perceived, notably by government officials who are part of the forum.In 2016, ELIXIR became a "Landmark Research Infrastructure" under ESFRI (ESFRI, 2016), meaning that it represents a major element of competitiveness of the European Research Area.Although being an ESFRI Landmark brings useful visibility to ELIXIR, it also requires us to engage with ESFRI processes such as an upcoming reporting exercise based on a common approach across research infrastructures that uses a set of key performance indicators (see Table 1

in ESFRI, 2019).
There again, the ESFRI indicators, and also the analogous 'core' indicators proposed by the Organisation for Economic Development and Cooperation (OECD) (see Table 2 in OECD, 2019), do not always work well against the specificities of a range of research infrastructures.Even "simple" numerical indicators relating to user numbers, disaggregated by sector (academic, industry), and to numbers of scientific publications can be tricky to put together in a distributed and virtual context, where resources are typically freely accessed via the internet without the need to apply or even register, and when the number of bioinformatics resources are in the several hundreds.The approach taken by ELIXIR has been to be proactive through engaging in the development of the ESFRI and OECD indicators, as well as creative in the development of alternative indicators and other evidence (e.g.simple narratives) that "make sense" in the context of ELIXIR.Table 2 (Supporting Information) lists these, along with mapping against the main impact areas of ELIXIR (Figure 1) and against ESFRI's objectives of greatest relevance for research infrastructures (under which ESFRI's set of indicators sit).
To give concrete examples of indicators, ELIXIR tracks user numbers of the ELIXIR Core Data Resources (Durinx et al., 2017), a set of resources (e.g.databases, curated knowledge bases) of fundamental importance to research in the life and biomedical sciences, via unique IP addresses (see Figure 1 of Drysdale et al., 2020), which amount to millions monthly.Providers of ELIXIR resources are increasingly developing dashboards to communicate their usage statistics in dynamic ways (e.g.single sign-on service: https://login.elixir-czech.org/stats;tools registry: https://bio.tools/stats;data analysis environment: https://stats.galaxyproject.eu),and work is ongoing to make this information more accessible and digestible by audiences that are either unfamiliar with ELIXIR and/or lack specific technical knowledge, as are found in certain groups of ELIXIR's stakeholders.
When it comes to monitoring the scientific value of resources under the ELIXIR umbrella, specifically their usefulness to users across the globe, the virtual and distributed nature of ELIXIR has required to build specific approaches.In the case of the set of ELIXIR Core Data Resources, this is done via inventorying the publications citing so called key articles describing the resources (in the thousands annually), along with detecting (via text-mining) mentions of the resources in the text of publications (in the tens of thousands annually) (Figure 2 of Drysdale et al., 2020).A similar approach is being cascaded to other ELIXIR resources outside of this collection (e.g.see citations/mentions widget on the ELIXIR Estonia website: https://elixir.ut.ee/).ELIXIR is also working to put in place systems to better monitor the use of bioinformatics infrastructure support (e.g. in terms of cloud computing) in scientific publications, via text-mining of key terms in the Acknowledgments sections.
Finally, text-mining of grant agreement numbers and other relevant search terms is also used by ELIXIR to measure its very own contribution to increasing the scientific knowledge linked to research related to developing the infrastructure and its resources (the impact area called resourcerelated knowledge in Figure 1).This last effort has the dual purpose of showing ELIXIR's scientific legacy, as well as reaffirming the scientific credentials of resource operators, who are still often assessed on publication records and grant awards if also leading scientific careers in parallel.

Challenge #4: Embracing the power of narratives supported by evidence
Whilst indicators are perhaps the obvious path to systematically and rigorously demonstrate public value, it is not always possible to develop and monitor them, due to lack of methodology, data, and/or capacity.In this case, narratives can be very suitable and effective alternatives to indicators, as supported by one-off evidence (e.g.self-standing studies) and/or ad-hoc (e.g.collections of examples) evidence.
For instance, ELIXIR showed that small and medium sized enterprises (SMEs) used some of its resources as business models for their mere existence (Roman Garcia et al., 2018).In line with this, preliminary results from another ELIXIR-funded study on the value of its resources to the private sector have revealed that more than 70% of respondents believed that, without data shared on open repositories, they would not be able to offer their product or resources.Furthermore, more than 90% stated that a product or resources has more features because of access to public registries, ontologies, and dictionaries shared on open repositories.ELIXIR plays a role in making these happen, via a dynamic industry engagement program whose activities are very efficiently presented as a suite of concrete examples (Lauer et al., 2019).Similarly, and related to ESFRI's indicator on 'facilitating international cooperation', ELIXIR regularly publishes its International Strategy (Martin et al., 2019a), which collates examples of activities undertaken to reinforce its global significance and impact, following a format (e.g.structured around Grand Challenges) that resonates with its intended policy sphere audience.

Challenge #5: Embedding impact evaluation in all activities is the next frontier
Few of the research infrastructures on the ESFRI roadmap (ESFRI, 2018) have dedicated and formally-trained staff on their payroll, with performance and impact evaluation tasks and skills forming the bulk of their job descriptions.Frequently, performance and impact evaluation and its results are spread over a number of roles in different teams (operations, finance, technical delivery, external relations and communications), in some cases taken on by busy senior staff members due to lack of suitable alternatives, and typically "learned on the job" reactive to external requests.In the case of distributed research infrastructures, such as ELIXIR, a range of arrangements are in place, reflecting local circumstances, capacities and needs.This diversity, plus the lack of visibility that comes with an organization spanning more than 20 countries, in turns makes it difficult to estimate the efforts and resources spent towards embedding impact evaluation in activities.
Recent years have seen the professionalization of research infrastructure staff, resulting in fewer occurrences of "scientific/technical" staff members (i.e.those developing and operating resources) also handling more specialized tasks such as linked to finance, administration, human resources and communications, and recruiting specifically for these roles when budgets allow.There still are, however, barriers to having economists and/or trained professionals in impact evaluation on the payroll in research infrastructures.Reasons include tight budgets, a misappreciation of the likely return-on-investment, and that it is possible to carry out one-off purely academic hypothesis-testing investigations with the help of external consultancies (Beagrie and Houghton, 2016;van Belle et al., 2018;Vignetti et al., 2020) rather than developing in-house capacity.The evidence generated by the latter will be state-of-the-art, yet usually focused in scope and soon outdated, thereby reducing its usefulness for quickly reacting to specific information requests from funders, and other monitoring processes led by external stakeholders (see examples in Table 1 in Supporting Information).Furthermore, it remains relatively expensive and hence hardly accessible to all research infrastructures and difficult to be regularly updated.
The approach taken by ELIXIR has been to build internal, practice-led, capacity in performance and impact evaluation through formal and informal means, ranging from mentoring by peers to formal courses and knowledge-exchange events, along with providing simple tools and worked examples to flatten the learning curve.EU-funded projects such as Research Infrastructures Impact Assessment Pathways (RI-PATHS) (Reid, 2021), which saw research infrastructures working closely with economists and impact evaluators to develop a simple toolkit (https: //ri-paths-tool.eu/en;Griniece et al., 2020) articulated around activity/outcome/impact rationales (rather than full-blown "theories of change" and "logical frameworks"), have been instrumental in informing and guiding these efforts.One main outcome of these cross-disciplinary and A "humble" list of actions (such as formal consultation responses, answers to information requests, and related activities such as position papers and events) to increase visibility of the infrastructure with, and its appreciation by, funders and the policy sphere more broadly could be categorized by target audience, and shown graphically as a rough indicator of effort (Figure 4) linked to the sustainability and policy influence impact areas (see Figure 1).Along the same line, a simple list of mentions of ELIXIR, its projects, resources and key achievements in policy documents, reports, guides for grantees, funding calls, etc, could be handled in a similar way, as a rough indicator of outcome, assuming no direct systematic causality (Figure 5).
As a matter of fact, no operational research infrastructure would need to be starting from scratch when it comes to putting in place monitoring systems or developing indicators.In the case of the ELIXIR Hub, much evidence of public value for ELIXIR as a whole already exists in some form or another, and only needs to be packaged and presented in a fit-for-purpose way for policy audiences.To systematize this effort and to tackle areas where more development effort is required, much can be achieved by half a Full Time Equivalent, working in collaboration with others across the research infrastructure.

LESSONS LEARNED ALONG THE WAY
Perhaps the main barrier to the uptake of systems to measure the public value of a research infrastructure is a limited understanding, and acceptance of approaches to evaluate performance and impact, within the infrastructures.Whilst performance and impact evaluation is well established and mainstream at all levels in the international aid and transportation sectors, it is not the case among the ground-forces of most research infrastructures, especially scientific ones where emphasis is usually put on reproducibility, and where qualitative approaches are rarely used and even met with a hint of scepticism.Endorsement by senior leadership of the organization is essential, and embedding impact evaluation in existing processes and systems will help with implementation and uptake, for instance through addition of impact-related questions to existing forms (e.g.proposals and end reports for ELIXIR-funded work).Finally, networking and communitybuilding opportunities between resource providers and impact evaluators, as happened during the RI-PATHS project, will go a long way in promoting cross-sectoral mutual understanding and appreciation.As the area of socio-economic impact of research infrastructures appears to become a field of study of its own, this field will more effectively mature as the bottom-up aggregations of practice-led knowledge and the purely academic hypothesis driven investigations become better connected.
The lessons learned shared here are based on the practice-led journey of one research infrastructure in the life sciences, yet we believe them to be sufficiently transferable and inspirational to other research infrastructures as they embark on their own journeys to demonstrate public value.To the funders of research infrastructures, our message would be that they have helped us get better, but they need to assess us on our individual and specific impact areas, rather than trying to develop one-size-fits-all systems that are likely to lack relevance and become meaningless as a result.To the ELIXIR Consortium, our message is an open invitation to join our community of practice in this journey, so that performance and impact evaluation becomes an additional part of our 'DNA', and we keep improving our resources and actions for public good.

A C K N O W L E D G M E N T S
We thank colleagues within and beyond ELIXIR who have contributed to informing and developing this work.We are particularly grateful to impact evaluators and economists who took part in the RI-PATHS (Research Infrastructures Impact Assessment Pathways) project, those running the Milan Summer School on Cost-Benefit Analysis of Investment projects, as well as the organizers of the XVIII Milan European Economy Workshop (25-26 June 2020, "The socio-economic impact of investment in science"), for fruitful discussions and guidance.Some of the work was carried out as part of projects funded by the European Union under Horizon 2020: ELIXIR-EXCELERATE (676559), RI-PATHS (777563), and ELIXIR-CONVERGE (871075).A. M. P. M., C. S. and F. de L. received support via an ELIXIR Staff Exchange project, and carried out some of the work as part of a field project for the Executive Masters in Management of Research Infrastructures (EMMRI).Finally, we acknowledge support from two infrastructure grants: LISBOA-01-0145-FEDER-022231-The BioData.ptResearch Infrastructure (Portugal); ELIXIR2 (270068), Research Council of Norway.Our indicators of visibility in the policy sphere were greatly improved by the use of the overton.iodatabase.

F
I G U R E 1 Main categories of direct impact for work funded by ELIXIR (not shown: Public awareness).These impact areas have proven to be a powerful and effective framework to communicate the value of ELIXIR's work to any type of audience, within and outside of the infrastructure.FAIR: Findable, Accessible, Interoperable, and Reusable [Colour figure can be viewed at wileyonlinelibrary.com]

F
Perception surveys are effective at capturing the effect of ELIXIR-funded flagship events, for instance on the development of personal outcomes linked to relationship capital (based on 113 respondents).These questions were asked to 150+ international attendees of the Biohackathon Europe 2019, during which bioinformaticians and programmers work together to advance technical projects.Each year, ELIXIR funds the costs of preparing and running this annual event (accommodation, catering, travel, social activities, merchandise, etc), with the exception of the staff time of the participants [Colour figure can be viewed at wileyonlinelibrary.com]F I G U R E 3 Visualization of connections established between ELIXIR Nodes, as facilitated by "commissioned services" (in 2019), as proxy for increased relationship capital.Commissioned services are a funding instrument used by ELIXIR to get experts in its national Nodes and EMBL-EBI working together on projects to implement the organization's five year Scientific Program.EMBL-EBI: European Molecular Biology Laboratory-European Bioinformatics Institute.Chord diagram based on code (R software) by G. Abel (https://guyabel.com/post/animated-directional-chord-diagrams/)[Colour figure can be viewed at wileyonlinelibrary.com]

F
Efforts to increase the visibility of ELIXIR with, and its appreciation by, funders and the policy sphere more broadly.Number of recorded actions (such as responses to consultations, answers to information requests and related activities such as positionings: temporal trend, and by target audience (pie chart).EC: European Commission; ESFRI: European Strategy Forum on Research Infrastructures; OECD: Organisation for Economic Development and Cooperation; UN: United Nations [Colour figure can be viewed at wileyonlinelibrary.com]F I G U R E 5 Number of mentions of ELIXIR, its projects, resources and key achievements, in policy documents, reports, guides for grantees, funding calls, and so on: temporal trend, and by source (pie chart).These are used as rough indicators of outcome (linked to efforts shown in Figure 4).EC: European Commission; ESFRI: European Strategy Forum on Research Infrastructures; OECD: Organisation for Economic Development and Cooperation; UN: United Nations [Colour figure can be viewed at wileyonlinelibrary.com] cross-sectoral interactions has been to empower research infrastructure staff in their efforts to evaluate and communicate the public value of their resources and their work.This has also meant looking at what was already monitored by the research infrastructure, but in a different light.