Data Collection

Data collection is the process of gathering and measuring information on targeted variables in an established system, which then enables one to answer relevant questions and evaluate outcomes. Data collection is a component of research in all fields of study including physical and social sciences, humanities, and business.

Regardless of the field of study or preference for defining data (quantitative or qualitative), accurate data collection is essential to maintain research integrity. The selection of appropriate data collection instruments (existing, modified, or newly developed) and delineated instructions for their correct use reduce the likelihood of errors.

Quantitative research is a research strategy that focuses on quantifying the collection and analysis of data. It is formed from a deductive approach where emphasis is placed on the testing of theory, shaped by empiricist and positivist philosophies.

Qualitative research relies on data obtained by the researcher from first-hand observation, interviews, questionnaires, focus groups, participant-observation, recordings made in natural settings, documents, and artefacts. 

A formal data collection process is necessary as it ensures that the data gathered are both defined and accurate. This way, subsequent decisions based on arguments embodied in the findings are made using valid data.The process provides both a baseline from which to measure and in certain cases an indication of what to improve.

There are 5 common data collection methods:

• closed-ended surveys and quizzes – refers to any question for which a researcher provides research participants with options from which to choose a response. Closed-ended questions are sometimes phrased as a statement which requires a response.

A closed-ended question contrasts with an open-ended question, which cannot easily be answered with specific information.  Examples of close-ended questions which may elicit a “yes” or “no” response include:

Were you born in 2010?

Is Lyon the capital of France?

Did you steal the money?

• open-ended surveys and questionnaires – a question that cannot be answered with a “yes” or “no” response, or with a static response. Open-ended questions are phrased as a statement which requires a longer response. The response can be compared to information that is already known to the questioner.  Examples of open-ended questions:

Tell me about your relationship with your supervisor.

How do you see your future?

Tell me about the children in this photograph.

• 1-on-1 interviews – according to Rossett (1999), interviews are the most common way of gathering performance analysis data.  Conducting one-on-one interviews with employees, managers, and executives allows them an opportunity to share ideas, clarify misconceptions, and to express their opinions, perspectives and points of view (Gilley, Eggland, & Gilley, 1989).

During a one-on-one interview, the interviewer asks questions of the interviewee in an attempt to seek opinions.  The one-on-one interview seeks to unveil data that is not observable, such as expertise or feelings.

• focus groups – a group interview involving a small number of demographically similar people. Their reactions to specific researcher-posed questions are studied. Focus groups are used in market research and studies of people’s political views. The discussions can be guided or open. They can concern a new product or something else. The idea is for the researcher to learn about the participants’ reactions.

• direct observation – used when you want to study the behaviour of a person or a group of people in a given situation. Sometimes the situation is natural and it is the observer who enters the environment of the observed. In other cases, the situation is recreated by the researchers, so that the observed is introduced into an artificial environment. .

The CIPD suggest that a clear, systematic and ongoing identification of how learning and development (L&D) needs relate to performance gaps is key in ensuring effective learning across an organisation.

Identifying learning and development needs starts with knowing the organisation’s current and future capability needs, and then assessment existing levels of skills, attitudes and knowledge.  This assessment can use formal and informal methods.  Such an analysis will allow decisions about what learning is needed at an individual, team or organisational level.  The gaps should be interpreted and prioritised within the wider organisational strategy.

Implementing an ongoing learning needs analysis (LNA) is different to a training needs analysis (TNA).  An LNA may be seen as a current or future health check on the skills, talent and capabilities of the organisation (or part of the organisation) and is carried out with multiple stakeholders.  It’s based on a systematic gathering of data and insights about employees’ capabilities and organisational demands for skills. Alongside an analysis of the implications of new and changed roles for changes in capabilities.  A TNA is more of a one-off event looking at the specific needs for a specific learning event.

The LNA process needs to flow from business strategy.  Its aim is to produce a plan to make sure there is sufficient capability to sustain current and future business performance.  It’s also vital to consider statutory and compliance requirements.

Analysis of learning and development needs can be done for:

• The whole organisation – to analyse the amount of types of learning needed to ensure that all employees have the right capabilities to perform in line with the organisation’s strategy
• As a specific department, project or work stream – new projects and opportunities require new ways of working or reorganisation, while restructuring also impacts.
• Individuals – linking personal L&D needs to those of the business, often carried out as part of a development review.

After planning the frequency, extent and nature of the analysis the next stage is to decide it can be collected.  Potential methods can include:

• Organisational data and intelligence – ‘mining’ the existing data that’s collected in the organisation is a great starting point.
• Formal interviews and / or focus groups with stakeholders.
• Informal conversations with stakeholders.
• Team meetings.
• Questionnaire-based or other surveys of managers, employees and their representatives.
• Pre-existing online data – i.e. management information systems (MIS) or virtual learning environment.
• Information and analysis from existing competence frameworks.
• Performance management data
• Documentations – i.e. business plans, objectives, work standards, job specifications.

A mix of all these together will give the best results.  Collating the information from the needs analysis will allow a number of outputs that can happen concurrently:

• A report of overall learning needs for the organisation or department – to form the basis of an L&D strategy or input to business planning processes.
• Prioritising the identified performance gaps – that is, where the gaps are most critical.
• Learning and development plans – once priorities and budgets are identified, the L&D team are able to set plans for learning solutions. 
• Personal development plans – plans for individual leaning, aligned with the resources available.
• Is a formal intervention needed? Many organisations say they offer a 70:20:10 approach to learning.  The needs analysis may support using job-related experiences (the 70%) or interaction with other (the 20%) rather than the formal elements (the 10%).

The outputs should be discussed and agreed with relevant stakeholders including senior managers, the learners and their line managers.

An article in People Matters, discusses the key trends in L&D and how the next generation of L&D means rethinking how organisations approach workforce skills and talent management. With the advancement of artificial intelligence (AI) and machine learning (ML), new learning needs have emerged, and the way we consume content has changed dramatically. We now have access to any learning at the click of a button. 

The smart capabilities derived from embedding AI and ML have permeated into our day-to-day lives, be it shopping online or streaming a movie. A global technology giant has also reimagined learning using AI augmentation to assist differently-abled employees. So, employing similar emerging technologies to curate personalized learning environments would have a striking impact on how L&D is viewed.

Anytime, anywhere learning model – employees today spend a significant amount of time traveling, in transit, or working flexible hours. Many of them embrace the routine of utilising their journeys by accessing video or audio learning modules on their mobile devices, increasing their productivity at work.  

Gamification and experiential learning – keeping a multigenerational workforce engaged is one of the biggest challenges facing organisations today and they are adopting gamified learning in different ways. Some implement partial gamification, which uses game-techniques like certification badges or rewards; whereas, some integrate their Learning Management Systems to personalise learning paths complete with leader boards and scores. This smart end-to-end solution motivates employees to elevate their skills but also inculcates accountability for their career and learning trajectories. Incorporating a measurement metric into the system also proves to be an effective mechanism to quantify the elusive L&D goals and ROI.

Prioritising soft skills training – many organizations overlook the importance of equipping their workforce, especially leaders and potential leaders, with basic, yet necessary, soft skills like communication, collaboration, and critical thinking. As organisations become more data literate, they are using the massive amounts of employee data available to them to enhance learning strategies and training curriculums for soft skill development as well. L&D leaders can understand learner behaviours and personalities to identify prospective leaders and fast-track their leadership development by filling these skill gaps required for the job.

We are in the early stages of collating and analysing L&D data.  We’ve looked at money spent.  We don’t have a budget set but at some point will, so it was important to understand current spend. 

We recently ran a e-questionnaire to look at individual needs and wants, mainly to highlight the L&D was a key business focus for the coming months but also to gauge employee interest.  The survey covered 270 people however only 37 people responded.   We did establish that those who did respond were interested in their career development and 87% would be happy to additional learning in their own time.  We hope to run this training again in years’ time to see if there is a change.

Internally we track our apprenticeship success.  The first chart below shows we have a 28% failure rate.  These are apprentices who did not finish the course.  When we analaysed our leavers we could see it was an issue in the recruitment process and outlining behavioural expectations.  By working with a recruitment provider we have been able to reduce this.

Most of our apprentices are covering level 2 and 3 qualifications but we have been able to provide some higher level qualifications in L&D and technical standards. We have found that recruiting engineering apprenticship has been successful in meeting our succession planning.  Only three of twelve apprentices have left the business.  The remainder whent on to secure permanet roles in the business and now as the older generation retire, they hold the key to business growth as we look towards investing in more lines and a new facility.

We also collate data from learners. This is used to improve the performance of facilitators by choosing their “weakest” area to work on.

Moving forward, building a bank of data is key to the success of our future as a team and a business. We are now planning 2022 strategies for further growth in a difficult labour market so lets take some time to understand what skills and resources we have on site, what we need and take a brave step forward.