How to decide whether on-site or online interviews are best when conducting an evaluation

The global shift to digital platforms during the COVID-19 pandemic transformed how we conduct our professional and personal lives, from virtual business meetings to online workouts and happy hours.

For those of us in the evaluation community, this era marked a significant pivot from traditional on-site interviews to online ones. The pandemic has now faded away, and we find ourselves at a crossroads. Evaluators are faced with a critical question: Should I stay or should I go? Should we continue to embrace the convenience of online interviews, or is there value in returning to on-site interviews in evaluations?

Evaluators working at the European Investment Bank Group’s Evaluation Division shared their recent experiences. Their feedback can help evaluators at other institutions make informed decisions about the interview formats that work best for different kinds of evaluations.

Comparing online and on-site experiences

Online and on-site interviews have pros and cons.

The immersive nature of on-site visits and face-to-face interviews offers insight into a project's context and results. Physically visiting a site allows evaluators to interact spontaneously with interviewees, such as through small talk in the hallways or on a drive to a site. It also enables them to better observe non-verbal cues, such as eye-contact during questioning. These interactions can often yield important information, and they are crucial for building trust. However, the logistical challenges, time commitment and environmental cost associated with travel cannot be overlooked.

Conversely, online interviews provide flexibility, inclusivity and efficiency, reducing the logistical burden and eliminating travel-related costs (budgetary and environmental). Moreover, the virtual format can be particularly advantageous for individuals who may feel uncomfortable in face-to-face settings. Some people might find online interviews to be less intimidating, encouraging them to open up and express themselves freely. Yet, the virtual approach can limit the depth of conversations between interviewers and interviewees, making it harder to capture non-verbal cues and to build trust.

What to consider

Following are a few things evaluators should consider when deciding between online and on-site interviews.

-- The evaluation topic and the role of interview data. The decision between online and on-site interviews hinges on the evaluation objective. On-site interviews are indispensable when they serve as the primary source of information for understanding whether projects were successful and the context surrounding their outcome.

EIB evaluators, for example, appreciated the value of face-to-face communication in a recent visit to sub-Saharan Africa. They were conducting a sectoral evaluation that covered three agriculture projects financed by the European Investment Bank. The on-site visit allowed evaluators to speak directly to different groups involved in the project. This included local banks that provided loans funded by the EIB and received training or guidance through technical support programmes, people who benefitted from the investments and local authorities.

Evaluators were also able to talk to representatives from other international financial institutions who worked in the country and sector, thus broadening their knowledge of the challenges and collaborative efforts in place.  This contributed to a deeper, more nuanced understanding of the governance framework surrounding initiatives that could not have been acquired from a distance.

Additionally, observing these projects firsthand allowed evaluators to see the tangible outcomes of investments. Conversations with people involved in and affected by the project resulted in a deeper understanding of the operating environment. The exchanges helped evaluators to better comprehend the full scope and impact of initiatives and provided a clearer picture of how financial support for projects spurred local development.

-- The reliability of data. The availability of reliable monitoring information can make online interviews a viable alternative, especially when data provide comprehensive insights into the project and its results.

EIB evaluators found this approach useful when looking at the effectiveness of the European Fund for Strategic Investments, for example. Conducting virtual interviews with investment fund managers gave them a comprehensive overview of the programme and its impact, without having to track down individual companies that received financing.

-- Practical constraints. Digital infrastructure, accessibility and logistical considerations should also be taken into account when deciding whether to do interviews on-site or online. EIB evaluations often benefit from physically visiting remote areas where companies and people received EIB funds. Yet practical limitations often necessitate online interviews, for example in cases where the location is either inaccessible or unsafe for visits.

The need for a flexible approach

EIB evaluators stress the importance of a balanced approach that takes into consideration the benefits of online and on-site interviews. When navigating these choices, evaluators need to determine what the evaluation is trying to achieve, the reliability of supplementary data and practical constraints. The choice between online and on-site interviews is complex, and it is influenced by the specific demands of each evaluation.

Understanding these complexities will help evaluators harness the benefits of online and on-site interviews, enabling them to adapt to different situations. The COVID-19 pandemic changed how we all work. Like everyone, evaluators need to rethink how we gather information that is essential to our jobs, striking the right balance between a “distance” and “in-person” approach.

Marla Hinkenhuis is a member of the European Investment Bank’s graduate programme, while Mónica Lledó Moreno and Milena Reinfeld are senior evaluation officers in the Evaluation Division, which conducts independent assessments of the EIB Group’s activities.
Find more about our work at http://www.eib.org/evaluation