Methodology

Methodology

Research Questions - March 2024

What pedagogical tools and processes are available to support novices to overcome barriers to participation in game coding processes?
How can game design patterns support the development of coding practices with novices?
How can learners build agency in an evolving community of game makers?

Conceptualising activity in this learning design

This section examines the use of activity theory to examine the data and help conceptualise the learning environment and processes.

NOTE - for similar work see [@barab_using_2002]

Choosing a suitable unit of analysis / germ cell & situating which activity systems is of primary value to explore

This process is outlined in chapter 4 & 5 - Move some of that here - Or drop? EXPLAIN IN JUST ENOUGH DETAIL HERE.

Levels of Abstraction in Activity Theory

LINK INSTEAD WITH ROGOFF - SET UP FOR CHAPTER 4

The educational activity can be seen as different scopes of activity

wider education
guided activity
specific actions and operations.

While game making can be seen superficially as the primary activity,

OTHER RESEARCHERS ARGUE?? I argue that a narrower objectives most vital unit of activity for participants.

Choosing smaller units of analysis

Blunden makes a pragmatic proposal to use the project as they key unit of activity [-@blunden_interdisciplinary_2010; -@blunden_collaborative_2014]

Activity, Actions and Operations

EXPLAIN IN JUST ENOUGH DETAIL HERE.

Following the terms and concepts used by Leontiev [-@leontiev_activity_2009],

Move summary of Activity, actions, operation (as per Barab here), link to GDP as a project / unity of activity.

The concept of activity as a primary unit for analysis to research community activity was outlined.

Analysis of activity system through conflicts and contradictions between elements

Engestrom [@engestrom_expansive_2001] explains the role of contradictions as build up of tensions between different elements of the activity system. His detailed analysis of work based activity identifies the role of contradictions in systems is the source of “change and development” []

While much of the literature focuses on larger systems of workplace analysis happening over years, the role of analysis of tensions in shorter interventions is also present in diverse research. [@barab_developing_2002; @barab_using_2002]

Unresolved contradictions develop into impasses or conflicts. These can serve as more urgent motivational factor driving an agentic response from participants in disfunctional systems. In particular work by Sannino on agency from a chat perspective frames the potential of such conflicts positively as a source of activist transformation. The following section explores agency in more detail.

Disambiguating terminology - Conflicts, tensions, contradictions, dilemmas

In activity theory tensions and contradictions are a vital source of change and evolution of new and ongoing activity systems.

In HCI and DBR and other design based research, similar concepts are are explored using a variety of different terms.

Even within schools of CHAT, terms are not set. To avoid the dangers of the vague use of the terms tensions and contractions [@karanasios_exploring_2017], this section outlines my interpretation based on the scope of this study.

While guided by the concept of analysing manifestations of systemic contradictions, [@cakir_contradictions_2022; @engestrom_discursive_2011] this study does not use that detailed framework.

NOTE - there is also the concept of congruencies - or strengths within activity systems. Where appropriate these systems strengths and positive alignments are helpful to note [@kamanga_contradictions_2021].

On a smaller scale, design tensions exist. These do not block progress completely but may disrupt some activities. Combinations of tensions can compound to create longer lasting and more serious conflicts if not resolved [Engestrom early]. The concept of levels of contradictions are helpful [@sannino_cultural-historical_2018] the forth being a conflict.

In Sannino’s later work on transformational agency TADS conflicts are examined - greater scope than tensions - thus larger shifts are implied.

Instrumental agency is relevant here to help examine and the work of learning designers to remove barriers, Transformational and authorial agency serves to explore the potential for learners to address and overcome tensions and conflicts in their own way.

SUMMARY OF DISAMBIGUATION In following chapters, following Kuuti [-@kuutti_activity_1995], I use the contradictions as a broad term wrapper encompassing concepts of tensions, barriers, breakdowns, conflicts.

In the following chapters I focus on tensions which help explore the research questions and which help inform the findings of later chapters. In particular I draw out more visible conflicts which threaten more serious disruption of activities.

Analysis of CHAT / DBR interventions over time

This section addresses concepts common to DBR, and early inteventions of CHAT.

While this is a contested topic, here I do not aim or a full analysis rather a working definition suitable for this study.

However, to explore tensions in a nutshell ….

Overall process DBR or CHAT or formative interventions - or expansive learning

Returning to the varied form of DBR / CHAT / Formative , I need to pick a terminology name.

Penuel [-@penuel_emerging_2014] chooses formative interventions as a suitable term, which is valid within CHAT circles.

Early limits of DBR addressed by expansive learning - and formative interventions

Critique of DBR often stems from limits in application. This section addresses some of this critique and introduces additional proposals to remedy early limits. This section identifies broad alignments in direction for CHAT perspective on DBR.

A principle of design-based research is that research participants also influence the ongoing design of the research [@barab_critical_2004; @cobb_design_2003; @downing-wilson_design_2011]. However in application this varies not a key part of all definitions [@easterday_design-based_2014].

For example, the context of experiments, particularly viewing the classroom as a controlled environment [@cole_design-based_2016]. In addition, lack of adaptability and mutuality in terms of taking on board the suggested adaptations of learners.

Engestrom [@engestrom_learning_2009] proposes a more rigorous examination of the dynamic nature of context with CHAT conceptions of Activity.

Adaptations including mutual appropriation from Cole’s work [@downing-wilson_design_2011] and formative interventions from Engeström’s work [@engestrom_design_2011]

Mutual appropriation highlighted the importance that researchers adapt to contextual factors including the practicalities of the setting.

Later work on DBR applied from a CHAT perspective has addressed limitations in this methodology [@oneill_understanding_2016].

Cakir and colleagues [-@cakir_contradictions_2022] draw on DBR but also include the wider constellations of activity systems working together on shared project.

Rajala and Cole echo limits [@rajala_utopian_2023] and argues for the use of FoK in design experiments in the form of funds of identity (FoI). The identity objects here be likened to the creation of artefacts in the game making process, and the evocative objects of Turkle and work of Papert.

Work on agency in change labs and more wider concepts of formative interventions are important to this study.

The role of the germ cell in rising to the concrete and thus expanding learning [@sannino_formative_2016; @engestrom_methodological_2014]

Expansive learning leads to the formation of a new, expanded object and pattern of activity oriented to the object. This process, known as ascending from the abstract to the concrete, involves the formation of a theoretical concept of the new activity, based on grasping and modeling the initial simple relationship, the “germ cell,” that gives rise to the new activity and generates its diverse concrete manifestations (Davydov, 1990).

From [@engestrom_design_2011]

This concept of shifting focus from individual activity to that of an expanded and shared object has parallels with more observational research on collective activity. (Rogoff) The key difference is the intervention of the researcher to create space, time and circumstances to facilitate the shift and production of new understandings of the activity in question.

When interogating the literature on expansive learning there is often the assumption of an activity already in progress which learners, often in work place settings, question as the first stage of the process.

How this maps onto a new activity in informal setting with novice learners is requires some clarification.

There are some areas that broadly align; the use of second stimulus, a thorny problem that requires a new solution, the move from individual action to a collective object and process.

Other areas are less aligned with existing models propsed by Engeestrom, CL, etc. The specific drive for participants to identify new tools,

Comparing Engestrom’s focus of research with that of Cole surrounding the 5th Dimension is a valid approach to shed light on the emergent nature of the experiment and the similar setting of the study in an informal after-school environment.

While, this study does not take a strong position on formative intervention / DBR, it is broadly aligned with the need to take into account learner-initiated innovation and the importance of broader contexts.

Designed systems: Learning environments are designed systems; be they existing systems like that of the classroom, or newly designed systems [@barnett_ecosystem_2019; @cole_design-based_2016].

This section has identified broad alignments in direction for CHAT perspective on DBR.

Exploring specific techniques of formative interventions - MAKE MORE TARGETED -

DROP FOR A MORE PERSONAL TAKE / SO COMPARATIVE TO THIS INTERVENTION? FIND MORE DIRECT PARALLELS? 5D FOR EXAMPLE

One of the most striking set of practical techniques are those used in Engeström’s take on Expansive Learning, specifically set of processes applied in a workplace setting called Change Laboratory (CL) [@engestrom_putting_2007]. The CL processes suits a formal workplace where there are existing concepts of Community and Division of Labour and Norms that have resulted in contradictions to be explored and resolved. EXPAND THIS WITH EXAMPLES?

As the game making activity for participants is novel there are few if any established community or working processes. Thus the full CL process is not applicable to this research intervention. In addition, the time intensive and explicitly analytical goals of CL are beyond the scope of commitment level and interest of the participants.

However, the expansive process and the use of transformational agency through double stimulation as a guiding technique offers significant opportunities. LINK THE CONCEPTS / PROCESS HERE.

For example, in and educational settings INSERT QUICK CASE STUDY

A complementary take on formative interventions also from a socio-cultural perspective is exemplified by the 5th Dimension (5D) project led by Michael Cole [@cole_design-based_2016]. This educational intervention used University support of volunteers, equipment and technical support in after-school settings in the San Diego area.

The intervention introduced a fictional narrative to create a shared ‘project’ for the different participants involved to promote a shared understanding of the work via intentional goal setting [@brown_cultural_2008].

In the 5D intervention, a narrative fiction of a ‘wizard’ to rapidly construct a shared sense of the guiding activities. In similar approach to Heathcote’s Mantle of the Expert, roles are created and breathed into life by the facilitator and affordances of the learning design to accelerate an adoption of principles of practice. (explored previously).

NOTE - IF USING PRACTICE THEN EXPLAIN IT IN REFERENCE TO AT - OPERAION / ACTION ETC

I am guided by the concepts behind the use of the models rather than drawn to an off the shelf application of CL processes or similar interventions. While some of the material used in CL and the fictional device of the 5D supports their primary tasks, others could be said to be support secondary considerations of navigating learning, establishing shared concepts and communities and reflecting on transformations in practices or understandings. These dimensions are sometimes explored in CHAT using the term of secondary stimuli used to facilitate and complement the primary task as part of the concept double stimulation.

In addition the concept of diversion from the primary task are relevant to the more playful approaches of participants and facilitators to make the learning process attractive [@engestrom_expansive_2013].

REWRITE - EXPLAIN AND EXPLORE SECONDARY STIMULI

Creating ideocultures as designed systems

From Cole.

The concept of ideocultures is useful when compared to expansive learning and presumption of existing objects.

Rogoff’s three planes of analysis of learning communities

Communities of practice and communities of learners

Communities of practice - legitimate peripheral participation boundaries and barriers to participation

(Continue to) refute or problematise dichotomy of learner led, teacher led approaches [@mascolo_beyond_2009].

Participation on community activities as a valid approach.

Cultural and social factors are vital to development process even if hidden in as school setting. Guided participation as a middle ground between instruction and discovery.

Teachers benefit from the concept of scaffolding, and designing learning environments and interventions to support learners to enter into a pattern of activity.

This has been applied in various forms, PBL, apprenticeship models, for example have guided participation as a base via different forms of scaffolding.

Funds of Knowledge

My study will be guided by understandings of how home discourses or ‘funds of knowledge’ (Moje et al., 2004; Moll et al., 1992, p. 3) can be drawn on as a resource by participants to facilitate learning in new environments.

COPIED AND PASTED BUT USE AS BASE - IF NOT INCLUDED IN INTRO The concept of Funds of Knowledge emerged from research within United States Latino communities. The term addresses the use of knowledge and skills from participation in activities outside school that teachers can build on to help classroom work and to support the motivation of learners. Researchers found that Latino home cultures, skills and traditions were hardly visible in mainstream school cultures. This resulted in a form of deficit thinking about the performance of these communities [@moll_funds_1992]. Research by the UK National Literacy Trust [-@picton_video_2020] of 11-16 year olds found that 96% percent of boys and 65.2% of girls play video games. This study shows that while there remains a disparity between genders, game playing is still very widespread and young people are unlikely to be part of a household where no games are played.

Game making allows children to draw on funds of knowledge in various ways, perhaps in the choice of the kind of game that is to be made, in the setting or subject matter or the style and aesthetics of audio and visual elements of the game. Teachers can also draw out attitudes and knowledge of game cultures and bring them into the learning environment in an inclusive way. In addition, knowledge of game design conventions can be used by teachers to exemplify coding concepts. For example, consider conditional coding constructs. If Pac-Man touches a ghost, then a player life is lost. Such structures are described as a game design patterns. Werner and colleagues [-@denner_using_2014] found that the use of design patterns and game mechanics when teaching novice coders can increase accessibility for learners due the concrete and relatable approach.

Problematising Funds of knowledge and funds of identity which may perpetuate status quo negatively

and update outlined here [@rajala_utopian_2023]

Possible funds of knowledge

NOTE - THIS PROBABLY NEEDS TO GO SOMEWHERE ELSE - TOO CONTEXT SPECIFIC

As explored in the introduction this study explores an area where several contexts meeting, family learning, coding cultures and informal science clubs.

Retro Gaming as a Fund of knowledge: retro gaming and associated nostalgia can provoke a positive reaction from both parents and children. This can be used to leverage interest in creative projects.

This study positions this interest and identification as a fund of knowledge which designers should be aware of and utilise to help address some of the affective barriers to taking part in coding processes.

What potential does that give to family game making?

NOTE - HOW IS THIS EXPLORE LATER? IS THIS REALLY A METHODOLOGY ASPECT - OR SOMETHING FOR THE INTRODUCTION OR THE DISCUSSION CHAPTER 6 ?

Conceptual tools associated with agency

agency as choice already explored in in 5D

Cultural mediation via artefacts [cole], as a base.

Look for tensions in ATs - and how learners overcome barriers or blockages, is an instrumental type of agency

TADS - On double stimulation and agency

NOTE - HOW MUCH TO GO INTO THIS HERE OR BRING IT UP LATER?

Conflicts and tensions between participant motives and different parts of activity system are sometimes revolved quickly through change, or remain more persistent and can be referred to as blockages.

Double stimulation refers to the use of tools and processes to overcome such conflicts awareness of these tools and active use of them engenders agency [@sannino_principle_2015]. While many studies have examined the process in professional settings there is a paucity of study in the area of “student-centered contexts of learning” [@isaac_cultural_2021, p. 93].

Blockages caused by contradictions are also analysed the fields of design and HCI with an aim of revising designs to remove them [@karanasios_moving_2021].

On affordances

AT’s relationship with affordances are explored here [@ba_erentsen_activity_2002]

In my analysi of

Other researchers have found the the scope and focus of affordances important to clarify [@wolff-piggott_activity_2016-1]. Focus particularly in terms of who the affordance is for.

Affordances have been misused, assigned magical properties, including agency of themselves [SUPPORT].

This study follows socio-cultural take that while objects have a forma of agency, affordances are not magic and become interesting to educators as part of activity stemming from human-initiated activity.

Nardi and colleagues give an overview regrounding Gibson’s concept of affordances as action possibilities in line with activity theory concepts of mediated action [@kaptelinin_affordances_2012]. Their focus is primarily on individual action.

While there is a The wealth of HCI literature in line with activity which discusses and applies concetps of affordances, the focus is often on the individual rather than on collective action [@kaptelinin_affordances_2012].

However, affordances are originally conceived of as part of activity, not separated from it [@ba_erentsen_activity_2002].

NOTE - more on affordances here from Nardi

Link with TADS? TADS and the associated metaphor of warping anchors is normally applied in settings of group action.

Data Collection

Summary Table of Data Collection

Online version here https://docs.google.com/document/d/1jSE4UCEC7IleHi-tNfPwmVNTLUpzY5H5-do3bkHcLvo/edit

Name	Details	Description	Processing before analysis
Screen capture data	72 recordings from 12 sessions	Captured in Flashback pro and stored as fbr	Transcoded to mp4 (involving loss of keystroke and mouse click data)
360 Video Data	9 recordings from 9 sessions in P2 and P3	Captured in 360 Gear camera on mini SD card	Transcoded via Samsung app & command line tool ffmpeg - see appendix 3.x
Journal entries	3 journals in book form and 50 pages of electronic notes	Books journals contained planning, immediate reflections on sessions. Electronics documents focused more on evaluation of evolving sessions and problems encountered by participants	n/a
Programme resources created	Diverse resources (see Ch. 4)	Evolved during each phase - kept in online spaces	These resources were altered for an external audience after the programme ended
Practitioner Interviews	4 x 90 min (average) interviews	Video and audio data recorded via a zoom meeting. Data stored as mp4 video files.	Transcribed using open source software, then speaker data added and split into time zones of 5 minutes
Participant Interviews	4 x 90 min (average) interviews	One collected via zoom meeting and stored as mp4 files. Three recorded via flashback pro with screen capture to capture exploration of games and assets created.	Transcoded to mp3 audio files. Transcribed using open source software, then speaker data added and split into time zones of 5 minutes

Justification of data collection and selection

Alignment with with socio cultural approach

The methodology of the data collection and analysis process is informed by socio-cultural approaches. If found the work of Rogoff in particular to research community oriented processes particularly illuminating

The process is a suitable approach for informal education.

As previously explored, the research questions invite a broad spectrum of analysis - cultural and social factors.

Here I adopt a naturalistic (is this the right word) research approach involves gathering data in situ to find and describe patterns of behaviour.

Large amounts of video data were being gathered, a challenge which is explored later in this chapter

Turning attention to the gathering and analysis of other data, such a rich set of data, will allow the cross referencing of the many sources.

While diversity of data invites drawing on diverse methods of analysis (3 planes, DBR approaches), ATs provide a foundation for the analysis of these diverse data.

Challenges of data collection and processing

While the rich set of data invites the cross referencing of the many sources, it also presents challenge on how to deal effectively with the large amounts of data being gathered. The use of a 360 camera reduces the amount of live video footage of interactions needed. Instead of multiple cameras pointed in different directions to capture participant interaction, participants are arranged in a square pointing inwards towards a central 360 degree camera.

360 Video Processing

The processing of video and audio data of both captured screens and 360 video data and high quality audio data involved several stages yielding advantages in capturing the gestures and facial expressions of potentially all participants in one recording. It does however present challenges. While this merits further exploration a brief summary of challenges is presented here.

The process involves large video files that are easily corrupted in transfer. While the files are split into 8 minute segments but also can be confusing

time navigation - also confusing - sometimes put up on the screen - but inconsistently
moving and keeping a track of data from week to week and over years is complex.
processing data requires special software - from samsung
viewing data on a normal vlc player - needs extra processing using spatial media

This whole process is so demanding in terms of careful file management. Making me create a linux command line toolbox which is included as a technical appendix.

https://docs.google.com/document/d/1Y7MsZDY8ofvls5XO7tztSu8KFdClJo09o3qpWOdkb2M/edit?usp=drive_web&ouid=114325803502752689876

Screen Capture Processing

I chose the software flashback pro primarily as it was intalled on and availble at low cost on home computers. Flashback saves data in a format. This include audio, video screen capture, learner keystrokes, mouse movements and optionally one or more webcams. I decided not to record on webcams for the following reasons:

intrusive feel - having the webcam on and the resulting red light on the screen, I felt was less conducive to feeling comfortable than one 360 camera in the middle of the room.
less resources - recording webcam on the laptop
reduced video file size of final data file
less complexity when processing data -

It not possible to code the natively created files directly. Thus need to be converted to mp4 to import into coding software like Nvivo. In this process the keyboard and mouse data is lost.

Learnings and recommendations on processing

The following learning and recommendations emerged from this process.

Prioritise displaying time - have an old school clock under the 360 camera. For example.

Check viability of data at each stage of processing and movement.

Data Analysis

My study draws on traditional ethnographic methods of my journal as participant observer, participant interviews and analysis of artefacts used and created. I also record the workshop area with a 360 degree video camera and capture of screen data and audio of the laptops used by participants.

One of the challenges of adopting a naturalistic research approach is to work with material gathered to find and describe patterns of behaviour.

Overall table and rational of data analysis

There is a table online here to summarise data analysis

https://docs.google.com/document/d/1jSE4UCEC7IleHi-tNfPwmVNTLUpzY5H5-do3bkHcLvo/edit#heading=h.hpqiq8h52efk

Stage	Technique	Data analysed & Tool used	Commentary
1	Making observations of activity based on 3 planes approach	Simultaneous view of screen capture and 360 footage in VLC player. Practitioner interviews in VLC player	VLC was used over Nvivo for technical reasons (see appendix 3.x) Observations stored in an excel spreadsheet with options for personal, interpersonal and cultural planes, notes on use of game design patterns and commentary related to stage one thematic coding process The process helped to surface tensions and barriers in activity which emerge in digital making in general and specifically in these sessions. And surface themes for thematic analysis This was undertaken partially for 9 number in 5 sessions and helped inform the smaller selection of data for stage 2
2 & 3	Transcription of activity and conversations	Simultaneous view of screen capture and 360 footage. Actions and extracts of conversations Recorded into a word document	Activity split into 5 minutes sections and recorded as a grid format
2	Testing Thematic Analysis	Video capture data and 360 data in Nvivo	This process was made complicated by the format of the data in video files and their extended length. Stage one coded based on design cycle stages, types of participant interactions and which game elements were being worked on by participants This was undertaken for x number of video files in x sessions
3	Revised Thematic Analysis	Video capture data and 360 data in Nvivo	This process was made complicated by the format of the data in video files and their extended length. Stage one coded based on design cycle stages, types of participant interactions and which game elements were being worked on by participants

Notes to allocate to a stage

Rational and approach of data analysis -

Video data analysis involved description and transcription of video data and thematic analysis of the contents.

Analysis was in different forms, and drew on additional forms of data gathering.

Broad dipping into sessions via 360 video, advantages, focusing on community interactions, disadvantages of
Transcription and annoation of video capture footage in parallel with 360 data
Coding of sections of video to both emergent themes and themes relating to the reseearch questions.

NOTE - Import more from the post of 2024- 05 - 08 - in diary.

Data Analysis Stage One

As outline in the table above, stage one involved initial broad analysis of td varied forms of data and recording observations in an excel sheet.

Rather than taking a grounded approach this data was analysed using 3 planes of sociocultural activity. The existing relevant framework saves time, especially as a more emergent approach was (in part ) in coding video data.

WHY NOT GROUNDED? OR TO WHAT EXTENT WAS IT GROUNDED? IN THE AREA OF INTERPERSONAL ACTIVITY.

3 planes approach - noting tensions and behaviours

RQx addresses the formation of a game-making community. This process is co-constituted by planners, facilitators, parents and children.

Following the approach of Morcom [@morcom_scaffolding_2014] I drew on Rogoff’s [-@rogoff_observing_1995] three planes of sociocultural activity used to help understand activity in a Community of Learners.

I created an excel spreadsheet and noted behaviour and tentions that emerged in transcription and coding.

This helped to get closer to the data and to :

start identify to thematic analysis of the stages of design with other learning models like PBL
to identify roles through repeated patterns of interactions, and contrast to Barron’s work
to reflect on the thesis questions

While the three planes provided a higher level framework, specifics of the data gave greater granularity and formed sub categories of coding

The advantages of drawing on funds of knowledge, and the activities that I used to do this, as well as connecting to wider communities are address well with the plane of culture, in the concept of Apprenticeship.

Justification of selecting sessions and sections for more detailed analysis

Then deciding on 8-10 video sessions warranting full coding of activity. This was based on the following factors, quality of recording, how much activity, richness of interactions, a range of diffent kinds of participant interactions.

Criteria for inclusion included both practical and theoretical considerations.

Good quality recording
Diversity of kinds of participants, ages especially
Diversity of participant actions in terms of tasks being worked on
Diversity of kinds of pair and peer interactions

A page of quick analysis of the files online here helped to shape that decision on what sections https://docs.google.com/document/d/1vpAIqHPg_Z0S5MWeS0rBTvxCL1N9NQButam4Ozq39Ek/edit#

Data Analysis Stage Two

Schema for initial thematic analysis in stage two was informed by literature review, early analysis of journal entries and emerging observations of data in stage one

Partial transcription/description of data

The partial transcription of video data formed a significant part of the data analysis. The large amount of video data recorded forced a prioritisation of data to be transcribed. It also forced a partial transcription process which was in part descriptive and in other parts involved a closer transcription of the words spoken by participants.

To aid the process of coding, I broke up analysis of in 5 minute sections. In addition, when these sections were described I imported data into nvivo and coded each of the 5 minute sections to the

This limited the accuracy of the process. In part, this element of the research was limited by the practicalities of working with this amount of data.

I priorised more data as this was needed to get a greater scope of activity to allow analysis of wider community activity and …

Partial coding of video at stage 2

Thematic analysis is commonly done in stages, each revision building on insights of the previous one.

While sstage one surfaced themes, stage two created a coding structure in nvido

Summary of thematic analysis - https://www.plymouth.ac.uk/uploads/production/document/path/18/18247/Kiger_and_Varpio__2020__Thematic_analysis_of_qualitative_data_AMEE_Guide_No_131.pdf

Video coded in segments in Nivivo - Stage 2 In this level, the rough coding involved by breaking up video timescales into 5 minute sections lessens the ability to make exact claims however for this study it opens up lines of initial questioning which can be explored in future studies or triangulated with other data in this study.

Describing different grades and pass overs of the video data.

quick analysis
sections coded and described in 5 minute chunks
themes of interest identified - in relation to evolving RQs as per DBR
some sections transcribed in more details based in themes, more detail on gesture analysis also added

What was excluded from initial coding and why

The following themes emerge as potential learning outcomes in the learning model, and were observed in interactions in stage one

Elements of Computational Thinking
Systems Thinking Concepts

However, they were note included at stage 2 as there is other extensive or focused work in these fields (previously explored in the literature review) on CT which this study which this study does not need to replicate.

Coding Participant Roles and Interactions.

Evolving notes on coding of roles and interactions are in an online document here.

Excel doc - https://docs.google.com/document/d/19PafC_w_7uObYL4v86IGeeFZTw2yKxjHfP_k85rHk_s/edit

Roles are different from interactions existing on a higher order of anaysis. Repeated patterns of interactions can be seen to develop into roles. Thus a more grounded approach may concentrate initially on interactions and develop this to become roles through analysis. However, many roles exist in other similar domains.

For example, the work of my own staring point when coding was guided by Barron and colleagues work on the roles of parents as learning partners [-@barron_parents_2009]. The roles are as follows: Teacher, Collaborator, Learning Broker, Resource Provider, Nontechnical Consultant, Employer, and Learner.

when coding I started with some preset codes from the literature for examples Barron’s codes on parent roles when helping digital projects. However, as coding progressed, I modified and extended these codes to better match the data I was working with

Specifically, the following were removed from Barron’s typology, parent as employer, non-technical consultant. Learning broker and resource provider were merged as some of the detail of those roles involve take place outside of the learning environment being analysed in coded video.

I judged it appropriate to and less complex to analyse and code the interaction between participants rather than code the behaviours by participant type.

Coding Design Cycle Stages

There are similar stage based approaches to creating a digital project. Firstly project based learning (PBL), then Inquiry based learning, finally design thinking. This design could adopt any one of these, however the creation of a digital product more closely resembles a design approach with testing stages of the created product.

The design process characteristics are broadly as follows; the process is often iterative, involves evaluation and separates planning from implementation.

Resnick in this area has written of a creative design spiral.

The model is similar to the ADDIE model from instructional system design: Analysis, Design, Development, Implementation and Evaluation. The discipline of teaching engineering also has a similar design stage cycle with many contesting variations [@winarno_steps_2020]. Engineering is Elementary project adapted from the ABET (Accreditation Board for Engineering and Technology) guidelines [@syukri_impact_2018] involves the following steps; Ask, Imagine, Plan, Create, Test, Improve, Share [@hester_engineering_2007].

It has been adopted by the area of computing is one used in a recent book on coding by Bers [@bers_coding_2021].

It is possible to be more specific when coding and include a wider terminology of technology and domain specific language. For example, debugging in the area of computer programming and playtesting in the domain of game making. I have had to weigh up the value of including these more specific terms and deciding to keep a more generic approach primarily to reduce complexity while undertaking video coding. Sub-categories add complexity and time to the process without necessarity adding a great deal.
ADD IN A JUSTIFICATION BASED ON HOW RESULTS PAN OUT.

Selection of vignettes for deeper analysis

Full transcription of key moments: and description of participant activity, activity on screen including simple gesture analysis

Used in chapters to orient the reader

Incorporating movement around the room in analysis

The 360 camera allowed for side by side analyis of the group, pair or individual working. Due to the layout these were cluster often around a laptop.

In the initial stages of transcription and description having this data was extremely useful to disambiguate spoken statements.

As a personal experience of describing and transcribing even thought it was a significant amount of work to synchronise and then to play both video sources side by side, the additional information really aided the description process in adding detail to the interpretations of motivations of participants.

For an example see the vignette in appendix 6.x (Suzanna and Olivia) chapter 6 which details the movement and behaviours of the child away from the computer as an example of legitimate peripheral activity.

Simple gesture analysis

In some exchanges, 360 data was vital to explain spoken word.

This helped clarify some things that were unclear from audio and screen capture data.

For example, deictic referencing, where unclear statements were clafied with pointing or other gestures was common. Such gesture were used often to clarify game movements on screen, or in one case a software procedure of cropping an image.

Analysis of the evolution of the programme resources created

There were two main types of resources created.

supporting teaching / facilitation RESOURCES
games and game and assets created

The documentation and other forms of support evolved during each phase -

I did not try to collect any personal notes that participants used for planning. And this was minimal for many participants.

Chapter four explores the evolutin of these resources using AT as a tool for analysis of the existing and emerging contradictions in activity.

Data Analysis Stage Three

After drafts of the emerging themes were in place, these were sued to create a revised coding structure and data was recoded more systematically. .

Integrate the thoughts here https://docs.google.com/document/d/1p6m20wehYkTeevj9QucpzKCR4NioELndg2rZeLWFK-c/edit#bookmark=id.k9kvu0182p6u

A revised processes of thematic analysis

What was really needed - Prioritising the selection of data and quick reading of it in nvivo

Complexities - thus rejecting comparative approach.

After data analysis in stage one. The following revisions were made in overall schema of coding.

While useful, I dropping design stages as a schema, for justification and implications see discussion in chapter six.

The wider helping interactions, again while useful had drawbacks. Firsly, they were so numerous. Secondly, given the existing work of a general scope of digital making by Barron, it was not clear what I would add in terms of contribution to knowledge beyond broadly confirming it in the scope of digital game making.

Instead, an area that was more targetted emerged from these areas.

<!– The analysis of these resources using the three planes involved a choice of some of the following

using a spread sheet and putting categories and notes in to columns
using a table in word to record notes on data in-situ –>

Coding use of GDPs as mediational strategies

This is explored in chapter 5

Coding impact on design/ processes on learner agency - boiled down -

REMOVED AS PROPOSAL BUT INSTEAD HOW TO ADDRESS THIS?

FIND ANOTHER FRAMEWORK FOR AGENCY AND CODE TO THAT.

Answering RQ - on pedagogical resources / agency / use of GDPs

THIS IMAGINED FOR CONCLUSION BUT ALSO CODE TO IT? OR ABOVE?

learners developing agency through personal expression of home and other Identities
agency through choice over pathway via GDPS
agency through flexible design cycle pathways, especially via community playtesting
developing agency through celebration of different game maker styles
authenticity of tool use, especially docs and access to Community
authenticity of audience - and role of drama to facilitate reflection

On Final Transcription/description of data

At this stage coding schemes were more mature.

Partial transcription of more video files -

The coding of each extract via a selction was the process. To better aid analysis via Nvivo selections

MORE TO COME HERE

Use of journal data

My journal entries contain notes and initial analysis of workshop sessions and resulting reflections on experimental design and facilitation. Post-workshop participant interviews will gather information to support analysis of specifics of in-session interactions and relevant detail of wider activity systems of home digital use. Digital artefacts created as part of the game making process include text-based games, pixel art, digital audio sound effects and background music.

The process of analysis of journal notes and resulting resources required a different approach from that of video data.

The following resources were examined

Book Journal pages
Digital journal entries directly in response to teaching sessions

- <!– - Linear support book and print outs to support participants in sessions

Additional 3M model resources to help navigation and reflection and gameful approaches –>

Journal Notes were used in the following ways: (perhaps code this)

as a reflective tool after sessions
as a note pad to plan resource creation
as a sketch pad to explore the relationship between Concepts being explored and existing game making frameworks
as a

The quantity of data prevented extensive and detailed coding using thematic analysis.

Instead, observations are triangulated with data from lit review to expose key themes and tensions.

Use of Interview data

Practitioner data used to inform themse for dvideo ANALYSIS

Participant interview data mostly used to confirm and triangulate with video data from making sessions

Practitioner interviews

Four practitioner interviews were a rich source of thinking to help develop and deepen my reflection on key issues, to develop lines of thinking and to expose and explore barriers to participation in projects, tactics to address them and tensions experienced by practitioners and participants in similar creative coding projects.

The interviews were semi-structured allowing me to ask follow up questions to gain more detail on areas of interest.

The chosen interviewees represented a certain diversity in audience however as these interview are limited in number, resulting data are also explore in relation to existing research and no broad claims are made from this data alone.

Participant interviews

These interviews took place at the end of between P3 and P4.

At times the resources that the

These interviews allowed triangulation and confirmation of data observered in video recordings.

Limitations in interview data

MOVE TO CONCLUSION In retrospect, shorter interviews at more regular intervals with participants would have been useful to answer

Data validity, replicability and reliability and decisions of what to not analyse

Data Validity - Triangulation of Data - THE THEORY

Draw on other chapters to represent overcoming of weaknesses of design-based approach.

Having a wide based on data allowed data triangulation e.g. find a dood example of finding confirmation in interview data

Limits of / problems with video data analysis

The choice of the videos to code more fully is rather skewed towards finding interesting data. Thus it may not be fully representative of the experience of all participants.

Choosing in a different way - around less interaction - would expose more barriers to participation to game making using the 3M model.

Only suitable pairs were chosen. This removes a lot of validity from any generalisation of claims that could be made.

Breaking of data into 5 min sections is practical but does not allow for some of the finer interactions and changes of behaviour to be accurately coded. There may be some value is coding some key interactions more carefully in more granular detail.

In summary, the coding process is more valuable as a tool to allow more qualitative reflection and analysis rather than a quantitative process as attempted above.

To make a quantitative approach work better X,Y and Z would need to be in place which was not suitable for this study because of A and B.

Discussion on evolving coding process - PERHAPS SHIFT TO LAST CHAPTER finally

The process of trying to decide a main focus was often tricky.

Take for example the process of Da in 2019-05-08 - where he was leading some fairly advanced work on researching and reading developer documentation and the constructing a new design pattern.

There are competing and overlapping intention and foci here for the participants.

Solve immediate problem - add new pattern
communicate cultural practice of finding, reading and adapting developer level documentation and help from forums.
give hands on experience to the notice coder

Most clearly there is an overlap here between the interpersonal working on a code problem collaboratively for educational purposes and the interactions with the wider cultural artefacts and practices from a professional environment.

Thus the process of writing this up was not neat. To overcome this issue…

In addition the personal plane was perhaps the most problematic to address given the data gathered. Even when participants were demonstrating their personal knowledge by sharing back or directly altering game code, this was as part of a social and cultural practice. Thus my interpretation involved ….

Generalisation of Research

Data Replicability

The elements of the design are online and included in appendices.

This study faces an issue of being culturally embodies, thus think descriptions of context are needed here [@hoadley_creating_2002].

The final chapter addresses the need to replicate this study in different contexts and the challenges involved in replicating the spontaneous emergent elements of specifics of this study.

As such chapter four can be conceptualised as a design narrative (see [@hoadley_creating_2002]) to address the common deficit of detail of the designed product and process. “the usual study presents a technology fully formed as if it had risen from the oceans like Venus herself”

” Would that all interventional research included this kind of rich description of the “treatment” so that one might infer whether the results were applicable elsewhere.”

Generalisation of Research

Generalisation is one of the guiding principles of DBR. While results are intertwined with context, designs should be assessed for generalisation to other settings.

Explore this in more detail - draw out a debate on how situated learning in this context - and wider debate is.

More than one iteration.
In different settings
Using different with different software

This is potentially complex to describe but worth it if wanting to talk about generalisation.

NOTE - This could be dropped or picked up again in the limitations section

The process of exploring this guided the development of the research questions.

Research Questions

How Research Questions Evolved

The process was driven by the activity of collaborative game making driven by practicalities, and my interests as explored in the first chapter.

As justified by DBR and formative interventions, the process was driven by a meeting of context and the needs / interests of myself as a researcher.

Possible themes that had potential but which were disguarded included;

Computation thinking & systems thinking
Aligning with Project based learning

Themes emerging include the alternation of abstract and concrete as a suggested learning journey, explored in my journal. However this seems to be explored in research on semantic waves with studies of more breadth so it did not seem a good idea to follow that thread in my thesis.

There was something about the formalisation of this process into graph form which put me off. I also realised that the frameworks that I spent a while trying to make accessible and gameful in the design, grouping and theming of their presentation, had strong similarities to concepts of abstract curriculum if explored through a semantic waves lens. Curriculum seemed too well a well trod path for a PhD thesis.

Instead the community elements of the learning experience regained focus during the process of coding. There were particularly interesting elements of the learning environment which contributed to notable behaviours.

The area of in-depth study of learning and wider behaviours in the development of coding. Work on roles by Barron and developed by Roque showed promise identifying helpful roles and reflecting them back to participants. Such reflection sessions were not always replicable with a general home education group or other non-formal setting. Could this implemented in the design, in the way reflective techniques were implemented in to games. Thus making it more playful/ gameful.

What research questions are answered by what data gathering and analysis - and described in what chapter?

What pedagogical tools and processes are available to support novices to overcome barriers to game coding and design?
How can learners build agency in an evolving community of game makers?
How can game design patterns support the development of coding practices with novices?

RQ1

Both barriers and tools / processes are explored in the literature review
Interviews with practitioners in chapter 4 guided the design in broad principles.
Barriers are explored in chapter 4 on evolution of the design
Tools and processes in results chapters 5 and 6.

RQ2

Interviews with participants some informal and some formal instructed directly adaptations to design
Analysis of video data

RQ3

Interviews with participants some informal and some formal instructed directly adaptations to design
Analysis of video data

Discussion on Methodology

Discussion

NOTE - WHAT KIND OF DISCUSSION WOULD BE NEEDED HERE? A RETURN TO POSSIBLE BARRIERS THAT THE METHODOLOGY AIMS TO OVER COME

Perhaps summarise the the context, and motivations of the above case studies are analysed in relation to the context of this study.

Authenticity - creating value

Barab and colleagues discuss the importance of creating value in the world in their ecosystem approach [@barnett_ecosystem_2019]. This mirrors authenticity in applied social approaches - PBL as applied method of - situated learning ontology.

Summary of design evolution

I conclude this chapter with a section that introduces the next chapter which is a rich description of the evolution of the learning design.

Describing the formative intervention of this research

The design of the research experiment and learning I started from a very open position and has evolved from several iterations of collaborative work with participants.

The experimental team consisted of Home Educating families . Learners acted as researcher participants to guide the next iteration of the game making program both directly and indirectly. Direct input was through requests and informal feedback and structured end-of-course interviews. Indirect input came from research data in the form on the games participants created, my research journal entries on my interactions with and observations of participants and recorded audio and video data of the participants and their computer screen capture.

Practical factors

Key contextual factors have facilitated and shaped development of the research design including:

Impact of research practicalities
The location at the university

The availability of resources potentially attractive to this audience.

For example, the ability to have that number of identical computers for this audience in particular drove participipation to be in a university setting.

and using this shape of working which was determined by technical reasons but which had an impact nevertheless on the ability for students to move around the room and the absence of a front for the teacher to teach from.

Motivating or other effect of being part of a research project (Eliza) may be present. However, this study does not make comparative claims.

On the details of the design

More than a design the result was a dynamic community which is analysed on several scopes of activity.

The following overview of activity use by stage focusing on the evolution of tools and processes concisely exploring the rationale for their selection or production. Fuller descriptions of the tools used at each stage and links to the online resources created are available as appendix 4.x

Phases and data Gathering

Name and Date	Description	Data Gathered
Oct 2017 - Mar 2018 Experimental Course	Participants started with a blank canvas and used Phaser.js within the Thimble code playground as an online coding environment.	Digital and paper journal notes Learning resources created
May 2018 - Dec 2018 Resource Development Stage	A series of one-off workshops, and development of a “half baked” game template, and supporting online and printable resources. At this stage I changed the code playground to glitch.com and started to use the onlne graphics editor Piskel.	Digital and paper journal notes Learning resources created
Jan 2019 - Dec 2019 Glitch Game Club	Two iterations of game making course of 5-6 weeks using Phaser.js, glitch.com and Piskel. The template and resources created in the previous stage were used as a starting point but continued to evolve.. The room was arranged to allow for both group activities and audio and video recording as per my research ethics application.	Digital and paper journal notes Learning resources created 360 Video recordings of all participants Screen capture and audio recordings of each participant Code and graphical assets created by participants as part of game making process
Jan 2020 - May 2020 Make Code Arcade	Two iterations of game making course of 5-6 weeks using MakeCode Arcade tool. The template, resources and facilitation approach was based closely on the previous stages using glitch.com. The first iteration allowed for full audio and video recording. The room used in the second iteration did not allow 360 video data to be gathered.	Digital and paper journal notes Learning resources created 360 Video recordings of all participants Screen capture and audio recordings of each participant Code and graphical assets created by participants as part of game making process

Phases and resources used

Move to appendix - But mention here.

Description	Starting Resources
P1: Oct 2017 - Dec 2018; Experimental Course	Participants started with no set plan or toolset and were asked to plan and make a game in two larger groups of 5-6 participants of mixed ages. After several weeks, a minimal incomplete starting game code template was introduced in response to student need. Then One-off workshops at Mozilla and Feral Vector conferences and to PGCE computer students and the creation of a “half baked” game template	Phaser 2.6.2 javascript library; Thimble code playground; online graphics editor Piskel; audio creation tools
P2: Jan - Feb 2019; Glitch Game Club	First iteration of game making course of 5-6 weeks. The template and resources created in the previous stage were used as a starting point but continued to evolve.	Phaser; Glitch, Piskel; updated game template; quick start cards; step-by-step tutorials; code examples
P3: May 2019; Glitch Game Club 2	Second iteration of game making course with additional drama and reflective elements	As per P3; drama scenario; interactive chat page in glitch
P4: Jan 2020 - September 2020; Make Code Arcade	Two iterations of game making course of 5-6 weeks using MakeCode Arcade tool.	Make Code Arcade (MCA) tool; MCA Template starting game; MCA quick start cards; MCA game pattern menu; MCA game pattern tutorials; Learning Dimensions Map

P1: Exploratory stage: Oct 2017 - Dec 2018

Phase one was an extended, exploratory series of game making sessions over several months. This stage was aimed not at collecting user data rather than building my competency as a facilitator. The process began with no pre-existing supporting materials. In response to need, I adapted a starting code template of a platform game and created various code examples to add functionality requested by participants. This template was adapted from an online tutorial to create a game of the platformer genre (see glossary), a move which helped to limit expectations of the technical complexity of their game. At the end of this stage participants showcased their games to students in the foyer of the MMU Brooks building. Generalised feedback from participants this stage included; a consensus that the process was too long; more guidance in the process of familiarising themselves the coding tools initially before undertaking ideation would be useful; group work was problematic given the lack of experience in this domain; and that the overall experience of working with code was difficult but rewarding (feedback notes are included as appendix 4.x). At the end of this phase I responded to this feedback and delivered three one-off workshops at conferences and community events. The compressed time frame sparked the use of new tools including a starting template and printable resources to accelerate the game-production process. The results in terms of engagement and impact on rapid personalisation of end products were very promising.

P3: Accelerated game making programme (5 weeks): Jan - Feb 2019

By the beginning of P2 most of the resources and activities were in place and relatively stable. There were some incremental evolutions in the template that stemmed from participant feedback and my reflections. Key additions at this stage to support the use of a template were a collection of tutorials and code examples illustrating key game features that had been requested by participants. The collection of game patterns were presented in a web page together with the starting template and a link to an online book collecting the process into step-by-step chapter based tutorials. The resources are explored in more depth in sections three and four below. A more complete description of the resources used in P2 and P3 are included in appendix 4.x.

P4: Adding a process drama to the game making programme: April - May 2019

P3 used the same toolset as P2 however in addition, drew on my previous work with the MMU Faculty of Education drama department [@caldwell_drama_2019] to create a simple drama process to give an external motivation and narrative to the creation of games using for a fictional audience of visiting aliens. The participant activity of these stages and tensions between system elements are discussed in detail in the remainder of this chapter and in following chapters.

On the emergent and mutual nature of the design and research process

The value of participant input to intervention-based research in the domain of technology use is well explored via participatory research [@iversen_computational_2018-1; @iivari_critical_2017], design-based approaches [@papavlasopoulou_exploring_2019; @barab_design-based_2004] and formative interventions [@cole_fifth_2006; @blunden_formative_2023]. In line with these research perspectives, the development of the tools and processes used by participants in this study emerged in response to my analysis of participant experience and their direct feedback. This process is particularly evident in the evolution of the learning design and resources. The trialling of the coding environment and starting templated occurred in various informal settings, over the course of two years and with input and concrete additions from parent volunteers, graduate trainee-teacher students, undergraduate students helpers University colleagues and with more indirect ongoing feedback from participating young people and their guardians.

While a full description of how this happened is beyond the remit of this chapter, this external input served several functions including: access to novel technical approaches proposed by parents and university volunteers; the surfacing of systemic conflicts hidden from my initial perception stemming from my pre-conceptions; observation of participant conflicts spurred the introduction of new tools to support development; and parental proficiency in adapting the learning environment to supporting their children leading to community-wide innovations in practice. My role as a facilitator included recognising and supporting the emergence of a repertoire of varied emergent practices in the responses of participants, and then to support these practices with suitable tools and resources. In this way, I support participants to build agency in their transformation of learning process.

While some of these dynamics are already well explored in work surrounding the 5th Dimension project [@cole_fifth_2006] (explored in the literature review), the presence of parents adds significant opportunities. I propose that this informal and exploratory setting with the participation of engaged parents, young people and their access to family learning dynamics and wider expertise provides potentially fertile ground in the generation of resources and domain practices. I explore the potential of this theme in more detail in the conclusion of this thesis.

Note on creation of learning and teaching resources

THIS IS COVERED IN THE NEXT CHATPER

The process of refining this reflection into learning and teaching resources was complex and iterative. The following paragraph illustrates the process using a case study? [PERHAPS LOOK AT THE ELEMENTs OF A GAME DIVISION? - FINAL DESTINATION - SPACE, MECHANICS, POLISH]

How does this methodology relate / add to existing models?

More material on Expansive Learning / Formative Inteventions / DBR

_posts/thesis_chapters/holding/methodology_notes_longform_pre_spring_2022.md

Transition to next Chapter

If needed.