Real Accounts Toolkit

2.4 Research team

2.1

Defining aims and target population for the project

As in the introduction, the financial diaries method is used to better understand the everyday management of incomes and expenditures by an individual or a household. As financial diaries are a mixed method and intensive methodology, it cannot generally be used at such scale to be representative of a specific population.

Instead, financial diary projects aim to understand a variety of experiences in greater depth. The specific aims of the project inform the methodology, particularly around recruitment for the project.

The Real Accounts project wanted to learn more about how people on low and medium incomes manage volatility in either income or expenditures. We decided to aim for 50 households with household incomes in the range of £10,000 - £42,000 per annum, where at least one person in the household was working.

We wanted to get a sense of the extent of volatility across low and medium incomes households, including those that would not normally consider themselves to have unpredictable income or expenditure, to capture all types of volatility. We therefore realised that it would be hard to include meaningful inclusion criteria about volatility in our recruitment materials. Rather, in sampling and selection, we focused our sampling on achieving as wide a variation as possible in terms of volatility (discussed further in the section below).

Another important factor to bear in mind is how fluid some of these categories were, especially around income, which changed frequently throughout the duration of the project, but also employment and household status. Similarly, we sometimes found out that what participants shared with us in the screening phases of the project was not a realistic assessment of their situation. So our sample was not strictly tied to the inclusion criteria.

Across the sample we aimed for a mix across gender, ethnicity and household status, including 1/3 single adult and 2/3 multi-adult households. We also sought a spread of urban, sub-urban and rural areas across the broad geographical zones covered by the three research partners, where the GCU team covered Glasgow and surroundings, the Aston team covered Midlands and the North West, and the Nest Insight team covered London and the South.

2.1 Defining aims and target
population for the project

2.4 Research team

2.2

Data collection infrastructure

At this stage, the research team should determine what infrastructure they will use to collect data in keeping with the aims of the project. There are two main parts to the data collection in financial diaries. The first is the quantitative tracking of household finances, which can involve:

Manual tracking – where participants and researchers work together to record transactions on a regular basis
Use of documentation – such as pay slips or benefit statements
Automated transaction tracking – as in the Real Accounts project, using our bespoke MyMoneyTracker platform (see Call Out 2.2.1 for more details).

The other part is the qualitative understanding of the household finances. Most studies use regular interviews over the duration of the study. These interviews provide the opportunity to record or confirm transactions as well as understanding patterns of financial management in more depth. Interviews can be held face-to-face, online, or on the phone. Some projects also use other methods such as focus groups or workshops. Note that meeting participants in person may need specific protocols to ensure the safety of researchers and participants.

The Real Accounts project decided to meet with households monthly to clarify the transactions captured in our monthly tracking and conduct ‘deep dives’ on different aspects of household financial experience which might not be covered through transaction tracking, such as, intra-household money management, financial relationships outside the household, use of debt and credit. We planned to use a combination of face-to-face and online meetings, as suited the participant. We also employed a series of workshops which happened after the end of data collection to engage participants in discussion of potential solutions to the challenges raised during the project.

2.1 Defining aims and target
population for the project

2.4 Research team

2.3

Compensation structures

Given the longitudinal nature of financial diary projects, an important aspect is how to compensate participants of their involvement. Most projects use a staggered approach to compensation to reward ongoing participation. Some projects vary or randomise the value or frequency of payments to minimise impact on financial management. The table below shows some of the different approaches to compensation used in financial diaries projects.

On the Real Accounts project, we decided to compensate participants for time and expenses incurred from participating in the study with a fixed amount (£40) after each monthly session. Households also received £70 after onboarding and setting up with MyMoneyTracker. Households where additional members also connected their accounts received a one-off £30 voucher to thank them for doing so. This means the highest total amount of compensations for households who completed the full 10 months of data collection received £470, plus a potential £30 voucher. We felt it was important for participants to be clear on the payments they could expect and when to build strong relationships with them over the course of the project.

While some financial diary projects have managed payments through cash payments given at in person meetings, remote participation requires a form of an electronic form of payment. Using a third-party platform also ensures that the research team do not have access to any bank account information.

2.1 Defining aims and target
population for the project

2.4 Research team

2.4

Research team

Many financial diaries make use of field researchers, who are employed to conduct the data collection. This is helpful where the project is taking place amongst hard-to-research communities, as field researchers who know the community can help facilitate access and support engagement throughout the project. The role of the research team in these cases is co-ordinating data collection across field researchers and reviewing the data collected.

In other projects, the research team also conduct the data collection. This is the approach we took on Real Accounts, as we did not anticipate issues with access and we felt there was a benefit in building relationships with our participants and staying close to the data. The research team comprised 9 researchers working across our three partner organisations, each looking after a number of households throughout the research process.

We found that preparing for the monthly meeting, conducting the meeting and completing the follow up work for each household took around 1 day’s work per month. However, it is helpful to be able to spread the work out as having too many interviews close together makes the work a lot harder. We recommended researchers lead up to 4-5 households each.

2.1 Defining aims and target
population for the project

2.4 Research team

2.5

Ethical approval

As financial diaries provide a window into the intimate lives of participating households, it is important that they are conducting to high ethical standards. Research conducted with academic partners will be expected to go through the University ethical approval process to ensure that the proposed research meets the required standard. This will include a detailed outline of the project methodology, processes for managing consent and data over the course of data collection, and versions of all documentation for participants.

Data storage and management is very important for financial diaries due to the potentially sensitive nature of the data. This can also prove complex where projects are collaborative and working across organisational boundaries, so needs some careful thought.

Bear in mind that approval will have to be revisited for any changes to the project.

The ethical approval for Real Accounts was led by our partners at Glasgow Caledonian University.

3 Initiation

3.1 Recruiting and screening

3.2 Creating research tools

3.3 Setting up project support tools

3.1

Recruiting and screening

Longitudinal research projects are often more difficult to recruit for than those with shorter timeframes. Financial diary projects also sometimes ask participants to do some work over and above attending research interviews, for example, where they have to record their transactions and other information in between. Even when using documentation or automated transaction tracking, there can be some apprehension about data sharing, which makes recruitment more challenging. Recruiting and screening participants can therefore take a lot of time and effort.

The Real Accounts project recruited primarily through our existing networks and used a professional recruiter to support this, especially amongst the harder to reach groups. We used a short online survey, which asked for basic information about participants and their households, as a device to screen for status against our inclusion criteria. This meant we could track the sample as recruitment evolved.

The survey was facilitated through a project website which contained all information about the project, presented in accessible format. This included video content and FAQs to engage potential participants.

Once participants had filled in the survey to express their interest, we found that it was useful to reachout to them for an informal conversation to make sure they understood what the project was about and to answer any questions or about the methodology, before we shared more detailed documentation. This also aided screening by confirming eligibility and addressing any concerns about what was required for the project.

3.1 Recruiting and screening

3.2 Creating research tools

3.3 Setting up project support tools

3.2

Creating research tools

At this stage, it is time to complete a full protocol for the research. This document should cover all data collection processes, step by step, and include detail such as topic guides for interviews. This will not only make sure that all those working on the project have a consistent approach but also act as a record of the project in future.

Tools such as templates and logs should also be created to be used by the research team, as well as setting up research team access to technology platforms such as for transaction tracking, data storage as required for the project.

The research protocol is an internal working document so can be added to as new things come up and are worked through on the project.

Some important things to consider at this stage are:

How to record regular meetings – for example, using audio recording and/or event logs, which summarise the key points. What equipment and software will be needed to facilitate this? You may also consider whether transcription of audio files will be needed.
How to categorise transactions – what level of detail needed for the project? How specific do you want to record things like loans (informal vs formal) or food costs (grocery shopping, takeaways, eating out).
What will you do if a participant stops responding? Participation often fluctuates in longitudinal projects and many things could happen during the project which affect their ability to respond. The research team could agree a timeframe to wait until chasing the participant, and after that time, an email or message prompting them to get in touch if they still want to be involved.

3.1 Recruiting and screening

3.2 Creating research tools

3.3 Setting up project support tools

3.3

Setting up project support tools

Outside of data collection, there are also tools needed to for project support activities.

First, a mechanism for tracking and paying participants. This needs to be secure and meet data protection requirements for the project. Real Accounts used a password protected tracker, which researchers would complete after each research interaction, with payments processed centrally once a week.

Second, a safeguarding process. In any research interaction participants may disclose issues concerning their wellbeing and this is especially true in long-term financial diary projects, even if not focussing on vulnerable groups. Having a project-specific safeguarding process to handle any disclosures and determine how to respond to them, such as signposting relevant support services,can help to support both participants and researchers. Examples of issues that were flagged during Real Accounts include gambling harm, financial scams and mental health challenges.

Finally, data management. The project might have many different data sources stored in different ways. While the research protocol will be clear on what should be stored and where, there should be a regular process to review this throughout the project.

4 Data collection

4.1

Onboarding participants

After participants are selected and before data collection starts, consent must be obtained. We completed electronically by sending participants the documents by email and asking them to send a signed consent form back to us.

In our onboarding meeting with participants, we revisited this consent to ensure participants were happy and went through the next steps involved in the research. Each household would be comprised of at least one participating adult. Where there were more than one adult in the household, there would be one lead participant and additional members were able to be involved if they wished.

We also used the onboarding session as a way to understand more about the participant through a financial journey exercise, invited them to tell us about their financial experiences up until now, through childhood and adulthood. This exercise proved useful to understand some of the long-term trajectories that shaped their financial lives over the course of the project.

4.2

Tracking transactions

The process for tracking and cleansing transaction data will be carried out through the project.

Real Accounts used real-time transaction tracking in the MyMoneyTracker platform, which provided an initial, automated categorisation of transaction type. We used the standard categories provided by the platform which covered most types of transactions, although some were missing and there was also a lot of room for interpretation, for example, when to us gift vs loan to describe an informal transaction between family members. We would recommend using an iterative approach around categories to account for new categories that emerge during the project.

Researchers would access the transactions for the previous month and go through them, updating any categories and flagging others for discussion with the participant. During the first part of the session, the researcher would ask about any transactions they weren’t clear on. The transaction data would then be updated after the session.

We found that many households had a lot of internal transfers, where they moved money between accounts. This was often due to purposeful processes of accounting where money was set aside in different pots for different purposes. Sometimes it was driven by volatility and the need to make unexpected payments. Identifying and understanding these internal transfers was an important part of our project.

We also collected household income every 3 months during the project. This is because our tracking did not always cover all sources of income, so these check points allowed us to understand the wider patterns of income and change during the period.

4.3

Conducting monthly sessions

Researchers met with participants monthly either face-to-face or remotely (online or by phone). Sessions were audio recorded using a separate device and uploaded to secure storage system after the session.

At the start of the session, research would confirm consent to record and also to continue participation in the project. Then they would invite the participant to review their month, and go through any transactions that needed clarifying.

The second part of the session would cover the ‘deep dive’, the topic of which would change each month, to understand aspects of financial experience not covered in the monthly tracking. The call out gives an example of the interview protocol and the topics covered through the deep dives.

4.3

Ongoing analysis work

Given the longitudinal nature of financial diaries projects, some of the analysis work will normally place alongside data collection.

For Real Accounts, this started after a monthly session took place. Researchers would complete an event log, a template created to record notes about the sessions, both from a substantive and methodological perspective. They would also record the session as complete in the tracker, to trigger compensation payment.

After updating the transactions following the participant’s feedback, researchers would create a monthly budget, which summarises the incomes and expenditures by category for the month. This would provide a view of change over the research period, sometimes revealing patterns that fed into later months’ preparations or themes for wider discussion.

Once a month, the team would meet for a collaborative analysis session. The purpose of these sessions was to discuss key topics and themes that were coming out of the research so far. We used a collaborative working tool to record notes from these sessions. Holding these sessions meant we were recording early and emerging themes in the moment and had a record to go back to later, once the project had ended.

5 Wrapping up

5.1 Offboarding participants

5.2 Finalising data

5.3 In depth analysis, outputs and
dissemination

5.1

Offboarding participants

Offboarding refers to the process of completing data collection by finalising the participants’ involvement. For Real Accounts, this meant conducting the last session and then deactivating their account connections in MyMoneyTracker.

Before the final monthly session, the researcher prepared the final month’s data, to be discussed with the participant in the first half of the session.

The second half of the session was used to conduct a wrap-up interview, which asked participants how they had found the experience of participating in the research.

After the final interview, each researcher finalised their work on the data and deactivated the participant’s connection in MyMoneyTracker, stopping any more data from being collected by the project. They confirmed this with the participant, who could thenfully delete the account they used to sign up to the platform.

5.1 Offboarding participants

5.2 Finalising data

5.3 In depth analysis, outputs and
dissemination

5.2

Finalising data

Finalising data refers to ensuring that data collection is complete to an adequate standard and that the data is stored in an accessible way, in accordance with the project’s data management requirements.

After data collection was complete, the team took some time to finalise the qualitative and quantitative data.

First, researchers finished off their event logs, transaction notes and monthly budgets for each participant. There were regular communications amongst the team to resolve issues and aim for consistency in approaches, for example, clarifying how transactions categories had been used.

Then work involved bringing all transaction data together into one database, undertaking validity checks and cleansing the data. This was a significant piece of work, even with the cleansing that researchers had already done with households’ transaction data. This database was then used to undertake quantitative analysis to support the in-depth analysis and outputs of the project.

Finally, the data from the project must be stored in a secure way, meeting the data protection requirements as set our earlier in the project. It must be accessible for researchers to continue work on analysis and outputs from the project. For Real Accounts, we put together a statement of access, to ensure that the data was being accessed and used in ways that met our data protection requirements.

5.1 Offboarding participants

5.2 Finalising data

5.3 In depth analysis, outputs and
dissemination

5.3

In depth analysis, outputs and dissemination

In-depth analysis refers todetailed and substantiveanalysis of the project data.

For Real Accounts, we undertook thison the basis of specific topics that were identified as interesting, either through the on-going analysis processes employed or in relation to external stakeholders. The analysis work was targeted to specific outputs, for example, impact briefs, conference presentations and academic papers.

Small groups of project team members would work together to conduct the analysis and produce the outputs, with opportunity for other team members to review and contribute to them. A publication strategy was employed by the project to ensure co-ordination of what data was being used in which publication to ensure consistency and avoid overlap.

Project outputs were disseminated primarily through Nest Insight’s communication channels with other partners sharing to their own networks, to reach stakeholders across industry, policy and academia.