Methodology

Methodology

Introduction

This chapter builds on the theoretical concepts explored in the previous chapter to explain how they are put into practice using a specific methodology to study the evolution of a game making community and the mutual creation of a new community-based pedagogy. The methodology of the data collection and analysis process is informed by cultural historical approaches.

The importance of context and cultural factors are particularly relevant for this intervention in a non-formal educational context with diverse stakeholder.

Driven by my dual role as a researcher and learning facilitator the process described here encorporates both research methodology and an overall structure of the evolving learning design. Thic

In terms of a reaseach process, this chapter describes a hybrid approach which while based on the principles of a formative intervention but incorporating other approaches where relevant. I justify this approach in the following sections.

Turning the learning design encountered by participants, I have worked with young learners, local families and undergraduate student helpers to co-create a game making design via iterative phases of rapidly evolving interventions most closely resembling a DBR process.

Here I cover the use of video recordings, participant interviews, and analysis of observational journal entries and resulting learning resources to achieve the systemic analysis previously outlined as typical of activity theory and CHAT-aligned design based reasearch (DBR).

Outline of chapter

INTERATE WITH ABOVE?

I will explore how this diversity helps triangulate findings.

this chapter describes:

an overview of the design stages of the study
data gathering and processing
data analysis explored in stages
comments on data validity, replicability and ethical considerations
analysis of broad AT system (MOVE?)

Summary of phases and resources of the study

The following sective provides an overview of activity use by phase briefly detailing the evolution of tools and processes. A following section outines the data gathering processes used at each stage before turn to the process of data analyis. Finding in other chapters, in particular the design narrative of Chapter 5, explore the rationale for the selection, creation and adaptation of both existing and novel tools and processes in each stage in line with the driving research questions. Fuller techincal descriptions of the tools and supporting resources used at each stage are beyond the scope of those questions and the limits of the structure of this thesis, fuller technical descriptions to and links to the online resources created are available as appendix 4.x. A summary of the delivery phases and one key development period are outlined in a graphical representation below in Fig 4.x.

{width=90%}

Fig. 4.x Summary of delivery phases and development periods

Before exploring the phases in more detail, an overview of the process of participant into into the learning design follows.

Overview of importance of participant input to learning design

EDIT DOWN?

The value of participant input to intervention-based research in the domain of technology use is well explored via participatory research [@iversen_computational_2018-1; @iivari_critical_2017], design-based approaches [@papavlasopoulou_exploring_2019; @barab_design-based_2004] and formative interventions [@cole_fifth_2006; @blunden_formative_2023]. In line with these research perspectives, the development of the tools and processes used by participants in this study emerged in response to my analysis of participant experience and their direct feedback.

This process is evident in the evolution of the learning design and resources. For example, the trialling of the coding environment and starting templated occurred over the course of two years and with input and concrete additions from parent volunteers, graduate trainee-teacher students, undergraduate students helpers University colleagues and with more indirect ongoing feedback from participating young people and their guardians. Input took varied forms. Direct input was through help requests made by participants and feedback and structured end-of-course interviews. Indirect input came from research data in the form on the games participants created, my research journal entries on my interactions with and observations of participants and recorded audio and video data of the participants and their computer screen capture.

My role as a facilitator included recognising and supporting the emergence of a repertoire of varied emergent practices in the responses of participants, and then to support these practices with suitable tools and resources. In this way, I support participants to build agency in their transformation of learning process. The following three chapters of research findings in part describe these transformations and emerging repetoires in more detail.

D1: Initial research and development - August and September 2017

From PhD proposal games for a read write world.

Phaser
Thimble

Before engaging with families in P1 in August 2017 I followed tutorials on Mozilla’s developer community website on writing games in javascript. I had used Mozilla’s Thimble code playground tool in earlier work teaching javascript, a snapshot of which is available as PGCE dissertation [@chesterman_webmaking_2015]. I discovered I could use Thimble, to house a working javascript based game. Mozilla promoted a professional, open source, javascript library for web game making called Phaser.js. At this time I also evaluated several online pixel graphics editors and chose Piskelapp.com.

P1: A phase of exploratory, co-creative game making - October 2017 to March 2018

Participants started with no set plan or toolset and were asked to plan and make a game in two larger groups of 5-6 participants of mixed ages. After several weeks, a minimal incomplete starting game code template was introduced in response to student need.

Phase one was an extended, exploratory series of game making sessions over several months. This stage was aimed not at collecting user data rather than building my competency as a facilitator. The process began with no pre-existing supporting materials. Several weeks in to the processes, I adapted a starting code template of a platform game and created various code examples to add functionality requested by participants. This template was adapted from an online tutorial to create a game of the platformer genre (see glossary), a move which helped to limit expectations of the technical complexity of their game. At the end of this stage participants showcased their games to students in the foyer of the MMU Brooks building.

Tools used: Phaser javascript library; Thimble code playground; online graphics editor Piskel; a varied set of other graphical and audio asset creation tools.

D2: Development Stage - May 2018 to December 2018

One-off workshops at Mozilla, Feral Vector and Manchester libraries events and to PGCE computer students helped focus and the creation of a “half baked” game template and supporting online and printable resources.

P2: Accelerated game making programme - 5 weeks in January to February 2019

Glitch Game Club First iteration of game making course of 5-6 weeks. The template and resources created in the previous stage were used as a starting point but continued to evolve.

Tools used: Phaser; Glitch code playground replaces Thimble, Piskel; a starter game template; quick start cards; step-by-step tutorials; code snippet examples.

NOT TRUE SEE LINK IN NOTES By the beginning of P2 most of the resources and activities were in place and relatively stable. There were some incremental evolutions in the template that stemmed from participant feedback and my reflections. Key additions at this stage to support the use of a template were a collection of tutorials and code examples illustrating key game features that had been requested by participants. The collection of game patterns were presented in a web page together with the starting template and a link to an online book collecting the process into step-by-step chapter based tutorials. The resources are explored in more depth in sections three and four below. A more complete description of the resources used in P2 and P3 are included in appendix 4.x.The experimental team consisted of Home Educating families . Learners acted as researcher participants to guide the next iteration of the game making program both directly and indirectly.

Direct input was through requests and informal feedback and structured end-of-course interviews. Indirect input came from research data in the form on the games participants created, my research journal entries on my interactions with and observations of participants and recorded audio and video data of the participants and their computer screen capture.

P3: Adding a process drama to the game making programme: 5 weeks in May 2019

P3 used the same toolset as P2 however in addition, drew on my previous work with the MMU Faculty of Education drama department [@caldwell_drama_2019] to create a simple drama process to give an external motivation and narrative to the creation of games using for a fictional audience of visiting aliens. The participant activity of these stages and tensions between system elements are discussed in detail in the remainder of this chapter and in following chapters.

Tools used: As per P2; a drama scenario; side missions; a code project to interact with aliens using markdown code.

P4: Adapting resources to MakeCode Arcade and more formal settings - November 2019 to September 2020

The last iterations of the learning design adopted the MakeCode Arcade tool as a new game authoring tool. This block based programming environment reduced participant coding errors allowing a greater focus on game making concepts.

In 2019 MakeCode Arcade was launched to make games using for novice coders via the use of a block coding systems similar to Scratch. I adapted resources with an aim that they could be used within secondary classrooms to support computing curriculum. To do this I added a learning dimensions map, a process which is explored in a later section of this chapter. I ran two iterations of game making course using MakeCode Arcade. The facilitation techniques and approach were adapted from those of P2 to facilitate the use of the block coding language [^1]. Tool use at this stage is also explored in a chapter in a collection on K-12 computing education [@chesterman_game_2023].

I also adapted and advanced resources with an aim that they could be used within secondary classrooms to support computing curriculum. To do this I added a learning dimensions map, a process which is explored in chapter.

Tools used: Make Code Arcade (MCA) tool; MCA Template starting game; MCA quick start cards; MCA game pattern menu; MCA game pattern tutorials ; Learning Dimensions Map.

Include footnote to online resources.

Data Collection

This section addresses data collection.

While the next chapter outlines the evolution of the learning intervention as a design narrative, the following table presents a summary of the phases involved to help situation this chapters description of data collection.

Phases and data Gathering

Name and Date	Description	Data Gathered
Phase 1 Oct 2017 - Mar 2018	Experimental Course: 13 participants started with a blank canvas and were asked to plan and make a game in two groups of mixed ages. A minimal incomplete starting game code template was introduced to support students after the course has started.	Digital and paper journal notes: Learning resources created
Phase 2 Jan - Feb 2019;	Glitch Game Club: 20 participants attend for 6 weeks. A new template are resources were introduced.	Digital and paper journal notes; Learning resources created; 360 Video recordings of all participants; Screen capture and audio recordings of each participant; Code and graphical assets created by participants as part of game making process
Phase 3 May 2019	Glitch Game Club 2: Second iteration of game making course of 4 weeks using Phaser.js and Glitch.com	Digital and paper journal notes;Learning resources created; 360 Video recordings of all participants; Screen capture and audio recordings of each participant; Code and graphical assets created by participants as part of game making process
Phase 4 Jan 2020 - May 2020	Rebel Game Makers (Make Code Arcade); Two iterations of game making course of 5-6 weeks using MakeCode Arcade tool. The template, resources and facilitation approach was based closely on the previous stages.	Digital and paper journal notes; Learning resources created; 360 Video recordings of all participants; Screen capture and audio recordings of each participant; Code and graphical assets created by participants as part of game making process

Table of methods of data collection

Name	Details	Description	Processing before analysis
Screen capture data	72 recordings from 12 sessions	Captured in Flashback pro and stored as fbr	Transcoded to mp4 (involving loss of keystroke and mouse click data)
360 Video Data	9 recordings from 9 sessions in P2 and P3	Captured in 360 Gear camera on mini SD card	Transcoded via Samsung app & command line tool ffmpeg - see appendix 3.x
Journal entries	3 journals in book form and 50 pages of electronic notes	Books journals contained planning, immediate reflections on sessions. Electronics documents focused more on evaluation of evolving sessions and problems encountered by participants	n/a
Programme resources created	Diverse resources (see Ch. 4)	Evolved during each phase - kept in online spaces	These resources were altered for an external audience after the programme ended
Practitioner Interviews	4 x 90 min (average) interviews	Video and audio data recorded via a zoom meeting. Data stored as mp4 video files.	Transcribed using open source software, then speaker data added and split into time zones of 5 minutes
Participant Interviews	4 x 90 min (average) interviews	One collected via zoom meeting and stored as mp4 files. Three recorded via flashback pro with screen capture to capture exploration of games and assets created.	Transcoded to mp3 audio files. Transcribed using open source software, then speaker data added and split into time zones of 5 minutes

Challenges of data collection and processing

While the rich set of data invites the cross referencing of the many sources, it also presents challenge on how to deal effectively with the large amounts of data being gathered. The use of a 360 camera reduces the amount of live video footage of interactions needed. Instead of multiple cameras pointed in different directions to capture participant interaction, participants are arranged in a square pointing inwards towards a central 360 degree camera.

360 Video Processing

The processing of video and audio data of both captured screens and 360 video data and high quality audio data involved several stages yielding advantages in capturing the gestures and facial expressions of potentially all participants in one recording. It does however present challenges. While this merits further exploration a brief summary of challenges is presented here.

The process involves large video files that are easily corrupted in transfer. While the files are split into 8 minute segments but also can be confusing

time navigation - also confusing - sometimes put up on the screen - but inconsistently
moving and keeping a track of data from week to week and over years is complex.
processing data requires special software - from samsung
viewing data on a normal vlc player - needs extra processing using spatial media

This whole process is so demanding in terms of careful file management. Making me create a linux command line toolbox which is included as a technical appendix. A draft of which is here

Screen Capture Processing

I chose the software flashback pro primarily as it was intalled on and availble at low cost on home computers. Flashback saves data in a format. This include audio, video screen capture, learner keystrokes, mouse movements and optionally one or more webcams. I decided not to record on webcams for the following reasons:

intrusive feel - having the webcam on and the resulting red light on the screen, I felt was less conducive to feeling comfortable than one 360 camera in the middle of the room.
less resources - recording webcam on the laptop
reduced video file size of final data file
less complexity when processing data -

It not possible to code the natively created files directly. Thus need to be converted to mp4 to import into coding software like Nvivo. In this process the keyboard and mouse data is lost.

Learnings and recommendations on processing

The following learning and recommendations emerged from this process.

Prioritise displaying time - have an old school clock under the 360 camera. For example.

Check viability of data at each stage of processing and movement.

Data Analysis

My study draws on traditional ethnographic methods of my journal as participant observer, participant interviews and analysis of artefacts used and created. I also record the workshop area with a 360 degree video camera and capture of screen data and audio of the laptops used by participants.

One of the challenges of adopting a naturalistic research approach is to work with material gathered to find and describe patterns of behaviour.

Overall table and rational of data analysis

There is a table online here to summarise data analysis

https://docs.google.com/document/d/1jSE4UCEC7IleHi-tNfPwmVNTLUpzY5H5-do3bkHcLvo/edit#heading=h.hpqiq8h52efk

Technique	Data analysed & Tool used	Commentary
Making observations of activity based on 3 planes approach	Simultaneous view of screen capture and 360 footage in VLC player. Practitioner interviews in VLC player 10 screen capture / 360 files from 8 group sessions analysed	VLC media player was used over Nvivo for technical reasons (see appendix 3.x) Observations stored in an excel spreadsheet with options for personal, interpersonal and cultural planes, notes on use of game design patterns and commentary related to an initial thematic coding suggestions. The process helped to surface tensions and barriers in activity. and helped inform the smaller subsequent selection of data.
Choosing and transcribing Vignettes	Simultaneous view of screen capture and 360 footage in VLC player. Practitioner interviews in VLC player 4 vignettes chosen and transcribed	A full transcription for a limited period of time with description and commentary on non-verbal aspects of movement and gesture. Getting deeper into the pair processes at play and a deep sense of what an individual may be experiencing.
Transcription of activity and conversations	Simultaneous view of screen capture and 360 footage. Actions and extracts of conversations Recorded into a word document	For the individual / pair their activity split into 5 minute sections and recorded in a word document time coded via a grid table format.
Exploratory Thematic Analysis	Video capture data and 360 data in Nvivo 8 screen capture / 360 files from 5 group sessions analysed	This process was made complicated by the format of the data in video files and their extended length. During this period activity was coded based on an evolving schema which included design cycle stages, types of participant interactions and which game elements were being worked on by participants.
Revised Thematic Analysis	Video capture data and 360 data in Nvivo 12 screen capture / 360 files from 5 group sessions analysed	A revised thematic analysis driven by the different use of game design patterns GDPs and dimensions of participant agency.

Notes to allocate to a stage

Rational and approach of data analysis -

Video data analysis involved description and transcription of video data and thematic analysis of the contents.

Analysis was in different forms, and drew on additional forms of data gathering.

Broad dipping into sessions via 360 video, advantages, focusing on community interactions, disadvantages of
Transcription and annoation of video capture footage in parallel with 360 data
Coding of sections of video to both emergent themes and themes relating to the reseearch questions.

NOTE - Import more from the post of 2024- 05 - 08 - in diary.

Data Analysis Stage One

Practitioner interviews

Practitioner data used in stage one to inform strategies and pedagogies for the initial design.

Four practitioner interviews were a rich source of thinking to help develop and deepen my reflection on key issues, to develop lines of thinking and to expose and explore barriers to participation in projects, tactics to address them and tensions experienced by practitioners and participants in similar creative coding projects.

The interviews were semi-structured allowing me to ask follow up questions to gain more detail on areas of interest.

The chosen interviewees represented a certain diversity in audience however as these interview are limited in number, resulting data are also explore in relation to existing research and no broad claims are made from this data alone.

These interviews are not explored greatly in the course of the analysis thesis as the key emergent themes were not explored in depth by those participants as while they were working in similar areas, they did not address game making explicitly. However, the interview process did has impacted in the overall direction of the work in by surfacing and focusing my attention on some key themes including: the helping roles of parents in digital technology; the role of competitions in after-schools settings, the value of surfacing and valuing the interests of partipants in digital making, and x.

A summary of the interviews is included as Appendix 4.x

S1 Video analysis

As outline in the table above, stage one involved initial broad analysis of the varied forms of data and recording observations in an excel sheet.

This data was analysed, at a broad scale, using 3 planes of sociocultural activity. The existing relevant framework saves time, especially as a more emergent approach was (in part ) in coding video data.

WHY NOT GROUNDED? OR TO WHAT EXTENT WAS IT GROUNDED? IN THE AREA OF INTERPERSONAL ACTIVITY.

3 planes approach - noting tensions and behaviours

RQx addresses the formation of a game-making community. This process is co-constituted by planners, facilitators, parents and children.

Following the approach of Morcom [@morcom_scaffolding_2014] I drew on Rogoff’s [-@rogoff_observing_1995] three planes of sociocultural activity used to help understand activity in a Community of Learners.

I created an excel spreadsheet and noted behaviour and tentions that emerged in transcription and coding.

This helped to get closer to the data and to :

start identify to thematic analysis of the stages of design with other learning models like PBL
to identify roles through repeated patterns of interactions, and contrast to Barron’s work
to reflect on the thesis questions

While the three planes provided a higher level framework, specifics of the data gave greater granularity and formed sub categories of coding used in the next stage of analysis.

The advantages of drawing on funds of knowledge, and the activities that I used to do this, as well as connecting to wider communities are address well with the plane of culture, in the concept of Apprenticeship.

S1 Use of journal data

My journal entries contain notes and initial analysis of workshop sessions and resulting reflections on experimental design and facilitation. Post-workshop participant interviews will gather information to support analysis of specifics of in-session interactions and relevant detail of wider activity systems of home digital use. Digital artefacts created as part of the game making process include text-based games, pixel art, digital audio sound effects and background music.

The process of analysis of journal notes and resulting resources required a different approach from that of video data.

The following resources were examined

Book Journal pages
Digital journal entries directly in response to teaching sessions

Journal Notes were used in the following ways: (perhaps code this)

as a reflective tool after sessions
as a note pad to plan resource creation
as a sketch pad to explore the relationship between Concepts being explored and existing game making frameworks
as a

The quantity of data prevented extensive and detailed coding using thematic analysis.

Instead, observations are triangulated with data from lit review to expose key themes and tensions.

Data Analysis Stage Two

Schema for initial thematic analysis in stage two was informed by literature review, early analysis of journal entries and emerging observations of data in stage one

S2 Video analysis - Transcription/description of data

The partial transcription of video data formed a significant part of the data analysis. The large amount of video data recorded forced a prioritisation of data to be transcribed. It also forced a partial transcription process which was in part descriptive and in other parts involved a closer transcription of the words spoken by participants.

To aid the process of coding, I broke up analysis of in 5 minute sections.

This limited the accuracy of the process. In part, this element of the research was limited by the practicalities of working with this amount of data.

I priorised more data as this was needed to get a greater scope of activity to allow analysis of wider community activity.

Justification of selecting sessions and sections for more detailed analysis

Then deciding on 8-10 video sessions warranting full coding of activity. This was based on the following factors, quality of recording, how much activity, richness of interactions, a range of diffent kinds of participant interactions.

Criteria for inclusion included both practical and theoretical considerations.

Good quality recording
Diversity of kinds of participants, ages especially
Diversity of participant actions in terms of tasks being worked on
Diversity of kinds of pair and peer interactions

A page of quick analysis of the files online here helped to shape that decision on what sections https://docs.google.com/document/d/1vpAIqHPg_Z0S5MWeS0rBTvxCL1N9NQButam4Ozq39Ek/edit#

S2 Video analysis - first pass of thematic analysis / coding

Thematic analysis is commonly done in stages, each revision building on insights of the previous one.

While sstage one surfaced themes, stage two created a coding structure in nvido

Summary of thematic analysis - https://www.plymouth.ac.uk/uploads/production/document/path/18/18247/Kiger_and_Varpio__2020__Thematic_analysis_of_qualitative_data_AMEE_Guide_No_131.pdf

Video coded in segments in Nivivo - Stage 2 In this level, the rough coding involved by breaking up video timescales into 5 minute sections lessens the ability to make exact claims however for this study it opens up lines of initial questioning which can be explored in future studies or triangulated with other data in this study.

Describing different grades and pass overs of the video data.

quick analysis
sections coded and described in 5 minute chunks
themes of interest identified - in relation to evolving RQs - to create evolving drafts of the emerging themes
some sections transcribed in more details based in themes, more detail on gesture analysis also added

NOTE - this document outlines thoughts on existing framworks which could inform coding process. https://docs.google.com/document/d/1ARce8w_31RXRZ_f5X1HyijJZkaGaX3K8y3QvKkaVgiA/edit#heading=h.3565tm76nud9

What themes were was considered but then excluded at s2

The following themes emerged as potential learning outcomes in the learning model, and were observed in interactions in stage one

Elements of Computational Thinking
Systems Thinking Concepts
Design cycle stages
Types of Gameplay Design Patterns

CT was not included at stage 2 as there is other extensive or focused work in these fields (previously explored in the literature review) on CT which this study which this study does not need to replicate.

Systems thinking was promising but not a proposed part of the learning design, and the data in P2 and P3 which became a chief focus of thus study was not as rich in this area as later P4 data.

Coding Design Cycle Stages & MDA / GDP framework

PERHAPS DROP OR AT LEAST REALLY EDIT DOWN HOW MUCH IS IN LR?

In Chapter 2, the potential of iterative design thinking as scaffolding approach was surfaced. I was interested in the promise of this.

It is possible to be more specific when coding and include a wider terminology of technology and domain specific language. For example, debugging in the area of computer programming and playtesting in the domain of game making. I have had to weigh up the value of including these more specific terms and deciding to keep a more generic approach primarily to reduce complexity while undertaking video coding. Sub-categories add complexity and time to the process without necessarity adding a great deal.
ADD IN A JUSTIFICATION BASED ON HOW RESULTS PAN OUT.

IS THIS STILL EXPLORED IN CH.4 - COULD IT MOVE HERE? I was interested in the question what could coding based on the different game elements that are being added to the game serves to illustrate. The process of learning game making through deepening your knowledge of game elements is explored by Schell [-@schell_art_2008]. As way of coding initially I considered the MDA framework (mechanics, aesthetics, dynamics) due to the focus on the user experience being taken into account.

This distinction was helpful in preparing resources as a navigational framework, as explored in chapters four and five. In that process I adapted the MDA framework to slimplify and match common features that had been suggested by participants. Four categories resulted: game mechanics, game space, game polish (aesthetics), game challenge (dynamics).

This categorisation prompted a line of questioning, would the application of different kinds of patterns, yield different results.

It did not yield particularly interesting insights. However, in coding I noted the use of the game design patterns as mediational strategies which proved to me much more fruitful. Thus I began to list the different mediational strategies that involved GDPs and this evolved into one of the main coding theme.

Final coding scheme (but this is stage 4)

Coding Participant Roles and Interactions.

Evolving notes on coding of roles and interactions are in an online document here.

Excel doc - https://docs.google.com/document/d/19PafC_w_7uObYL4v86IGeeFZTw2yKxjHfP_k85rHk_s/edit

Roles are different from interactions existing on a higher order of anaysis. Repeated patterns of interactions can be seen to develop into roles. Thus a more grounded approach may concentrate initially on interactions and develop this to become roles through analysis. However, many roles exist in other similar domains.

For example, the work of my own staring point when coding was guided by Barron and colleagues work on the roles of parents as learning partners [-@barron_parents_2009]. The roles are as follows: Teacher, Collaborator, Learning Broker, Resource Provider, Nontechnical Consultant, Employer, and Learner.

when coding I started with some preset codes from the literature for examples Barron’s codes on parent roles when helping digital projects. However, as coding progressed, I modified and extended these codes to better match the data I was working with

Specifically, the following were removed from Barron’s typology, parent as employer, non-technical consultant. Learning broker and resource provider were merged as some of the detail of those roles involve take place outside of the learning environment being analysed in coded video.

I judged it appropriate to and less complex to analyse and code the interaction between participants rather than code the behaviours by participant type.

Selection of vignettes for deeper analysis

Full transcription of key moments: and description of participant activity, activity on screen including simple gesture analysis

Used in chapters to orient the reader

Incorporating movement around the room in analysis

The 360 camera allowed for side by side analyis of the group, pair or individual working. Due to the layout these were cluster often around a laptop.

In the initial stages of transcription and description having this data was extremely useful to disambiguate spoken statements.

As a personal experience of describing and transcribing even thought it was a significant amount of work to synchronise and then to play both video sources side by side, the additional information really aided the description process in adding detail to the interpretations of motivations of participants.

For an example see the vignette in appendix 6.x (Suzanna and Olivia) chapter 6 which details the movement and behaviours of the child away from the computer as an example of legitimate peripheral activity.

Simple gesture analysis

In some exchanges, 360 data was vital to explain spoken word.

This helped clarify some things that were unclear from audio and screen capture data.

For example, deictic referencing, where unclear statements were clafied with pointing or other gestures was common. Such gesture were used often to clarify game movements on screen, or in one case a software procedure of cropping an image.

Analysis of the evolution of the programme resources created

There were two main types of resources created.

supporting teaching / facilitation RESOURCES
games and game and assets created

The documentation and other forms of support evolved during each phase -

I did not try to collect any personal notes that participants used for planning. And this was minimal for many participants.

Chapter four explores the evolution of these resources using AT as a tool for analysis of the existing and emerging contradictions in activity.

Data Analysis Stage Three

After drafts of the emerging themes were in place, these were sued to create a revised coding structure and data was recoded more systematically. .

Integrate the thoughts here https://docs.google.com/document/d/1p6m20wehYkTeevj9QucpzKCR4NioELndg2rZeLWFK-c/edit#bookmark=id.k9kvu0182p6u

STAGE 3 - In the final stage, previous transcripts with notes were imported data into nvivo and coded each of the 5 minute sections to the final thematic analysis.

Final coding schema for thematic analysis

Non GDP

Design Behaviours and adaptations - Drama related behaviours - Reflection in Role - side missions
Emergent aspects of idioculture
- authentic technologies as empowering
- self driven adaptations
- community playtesting
- self playtesting
Secondary tools
- using online code snippets
- using online or printed code tutorials
Primary tools
- Audio toolset
- Graphical tool use
- Template in Glitch
in personal appropriation of concepts
assisting project navigation
scaffolding ideation processes
aiding division of labour
GDPs in technical processes
facilitating designing for others
GDPs used as concepts in playtesting
playtesting used to propagate GDPs
facilitating use of home knowledge via GDPS

A revised processes of thematic analysis

What was really needed - Prioritising the selection of data and quick reading of it in nvivo

Complexities - thus rejecting comparative approach.

After data analysis in stage one. The following revisions were made in overall schema of coding.

While useful, I dropping design stages as a schema, for justification and implications see discussion in chapter six.

The wider helping interactions, again while useful had drawbacks. Firsly, they were so numerous. Secondly, given the existing work of a general scope of digital making by Barron, it was not clear what I would add in terms of contribution to knowledge beyond broadly confirming it in the scope of digital game making.

Instead, an area that was more targetted emerged from these areas.

<!– The analysis of these resources using the three planes involved a choice of some of the following

using a spread sheet and putting categories and notes in to columns
using a table in word to record notes on data in-situ –>

Coding use of GDPs as mediational strategies

This is explored in chapter 5

Coding impact on design/ processes on learner agency - boiled down -

REMOVED AS PROPOSAL BUT INSTEAD HOW TO ADDRESS THIS?

FIND ANOTHER FRAMEWORK FOR AGENCY AND CODE TO THAT.

Answering RQ - on pedagogical resources / agency / use of GDPs

THIS IMAGINED FOR CONCLUSION BUT ALSO CODE TO IT? OR ABOVE?

learners developing agency through personal expression of home and other Identities
agency through choice over pathway via GDPS
agency through flexible design cycle pathways, especially via community playtesting
developing agency through celebration of different game maker styles
authenticity of tool use, especially docs and access to Community
authenticity of audience - and role of drama to facilitate reflection

On Final Transcription/description of data

At this stage coding schemes were more mature.

Partial transcription of more video files -

The coding of each extract via a selction was the process. To better aid analysis via Nvivo selections

MORE TO COME HERE

Use of Interview data

Participant interview data mostly used to confirm and triangulate with video data from making sessions

ANALYSIS

Participant interviews

These interviews took place at the end of between P3 and P4.

At times the resources that the

These interviews allowed triangulation and confirmation of data observered in video recordings.

Limitations in interview data

MOVE TO CONCLUSION In retrospect, shorter interviews at more regular intervals with participants would have been useful to answer

Revisiting the research questions based on the evolution of the learning design.

Bakker offers guidance in the formulation of research questions in DBR studies which are relevant to this thesis [@bakker_design_2018]. He promotes HOW and WHAT CONSIDERATIONS formulation of question to be represent the exploratory nature of DBR approaches and to increase the possibility of possible generalisation without over promising in terms of projecting beyond the embedded nature of the findings as embedded in the context of the study.

Because the process of both formative intervention and DBR is exploratory with no set starting hypothesis, the process of formulation of RQs necessary one which is revisited as the process unfolds. This section considers some of those revisions as a way to futher explore detail of that evolutionary process.

RQ2 - Shift from family impact to broader quetions

A key driver of my research was to explore the potential to draw on family experience in learning activities by working with families to make games together. As the study evolved, the family dimension of interaction was only one of several which emerged as significant. However, it would be challenging to separate out the family elements from others as the social resource created by families working together proved to be significant. Thus additional behaviours between young people and supporting behaviours from student helpers may have evolved in a sense benefiting from this base platform. Thus it makes sense to widen the scope of one of the research questions beyond impact of family involvement on a game making process to instead to focus on a diverse and complex picture of the development of agency as a community, as outlined in RQ2.

How can learners build agency in an evolving community of game makers?

The remaining research questions guiding this study are broadly aimed at developing greater understanding existing and developing new the supporting mechanics of game making as a educational process, with a particular focus on the development of agency within a CHAT perspective.

What pedagogical tools and processes are available to support novices to overcome barriers to game coding and design?
How can game design patterns support the development of coding practices with novices?

Because it suited my working practices as an educational facilitator, and in line with DBR approaches the initial design evolved from phase to phase and relied heavily on input from participants and a wider, online, open source digital production-education community [] Mozilla link / Sefton Green.

The hybrid of DBR and formative methodology was chosen for its iterative, collaborative approach, which aligns with the study’s ethos of a detailed and systemic exploration of the expansion of the object of the game making pedagogy in an authentic and complex context.

What research questions are answered by what data gathering and analysis processes and stage

AT WHAT STAGE? Thematic analysis of video interactions and allow for a detailed examination of the social and communicative aspects identified in the research questions.

RQ1

Both barriers and tools / processes are explored in the literature review
Interviews with practitioners in chapter 4 guided the design in broad principles.
Barriers are explored in chapter 4 on evolution of the design
Tools and processes in results chapters 5 and 6.

RQ2

Interviews with participants some informal and some formal instructed directly adaptations to design
Analysis of video data

RQ3

Interviews with participants some informal and some formal instructed directly adaptations to design
Analysis of video data

Ethical considerations and processes

Ethical approval for this study was obtained from ESRI ethics process using the ETHOS system to ensure that all research activities comply with the guidelines for research involving human participants. Inclusion criteria included being part of a family (including guardians) unit with the ability to travel to the Manchester Metropolitan university campus campus. Younger family members will be between 7 and 13 years of age. All participants will have an interest in learning to make digital games together as an educational project. The project requires that they have literacy abilities to be able to read and write English.

Participants were be identified by involvement in activities linked to Manchester Met’s EdLab programme and that of Coder Dojo activities via an open call on email to interested parties. In addition to address a wider diversity of participants in terms of to Alternative arrangements may involve an open call to the Greater Manchester Home Education Network internet forum. In the ethics application I noted that participants may be excluded from studies if there is a over-representation of one gender or ethnicity over another. In practice all applicants were accommodated as the gender and ethnicity balance was broadly representative of the home education groups in the local area.

Given the involvement of three different participant groups of university students, parents, and children, I adapted assent forms, participant information sheets and processes to requirements of each group. For parents, a detailed assent form explained the study’s purpose, how data from children would be collected and managed, and the rights of parents, including their ability to withdraw themselves and their child at any time. For children, a simplified assent form was created using age-appropriate language. This form explained the study in simple terms, emphasising that participation was voluntary and that they could withdraw at any time without negative consequences. Written assent from the children was obtained as part of an informal discussion involving both children and parents. University students were also provided with a assent form explaining the study’s purpose, their rights (including voluntary participation and the right to withdraw without consequence), and data usage. They were given the opportunity to review the form and ask questions before providing written consent.

To ensure respect for the autonomy of all participants, particularly children, alternative activities were offered as part of the recruitment process for those who did not wish to participate in the study. This was important in settings where families may have felt a pressure to take part in research in order to benefit from the social or educational opportunities.

It was not possible to anonymise video, audio recordings at the point of collection, however is has been anonymised in the process of data analysis through the use of pseudonyms. The raw video data and additional data from the project has been stored on a password protected external computer drive, separate from the research data. Only the lead researcher had direct access although the ethics approval process gave project supervisors may be given access on request. I took care to not allow any personal data into the public realm as part of the research on analysis process.

As the study involved children additional safeguards were in place. All university students assisting with the study, as well as myself as the primary facilitator, had undergone Disclosure and Barring Service (DBS) checks as part of the EdLab studies programme. Activities involving children were conducted under the supervision of a parent or guardian, who could withdraw the child from the study at any time. Children were reminded, in terms they could understand, that they could opt out at any time.

Data validity, reliability and generalisability

The issues addressed in this section relate not only to the guidance methodology receives from the theoretical underpinnings explored in the last chapter but also the concerns of the driving researcher questions.

Issues of reliability and validity in research are significantly complicated when viewed through a post-objectivist critical stance, as exemplified in the previous chapter by Stetsenko’s transformative activist stance, which challenges positivist paradigms of research as a neutral endeavour in terms of values of those undertaking it [@stetsenko2014transfromative]. While, a full investigation of the Stetsenko and other’s rejection of an objectivist research approach is beyond the remit of this chapter, it supports the inclusion of my own auto-ethnographic journal as a valid source of data.

Data Validity

In general terms data validity refers to match between the findings of the study and what is happening in in practice. Turning to a more established exploration of data validity, Bakker’s [-@bakker_design_2018, p.88] exploration of the issues of data validity in DBR is well suited to the diverse data gathering methods of this study.

The strategies in this study are broadly, data triangulation, the use of rich descriptions, and a process of checking your interpretations of activities with participants (member checking)

Use of rich descriptions

Hoadley calls for rich descriptions of the learning design, a process which is applied in this thesis as a design narrative in Chapter 5.

Similarly, rich descriptions of the interactions of participants with the learning design, their own project creations and each other are included in the data results in small extracts, or where appropriate more extended vignettes. These allow for a fuller exploration of the learning processes in a richer context. Some details of interaction are needed to give other scholars and practitioners a full picture of interactions. The descriptions of facial reactions or body language in the vignette descriptions serve this end.

Data triangulation

Having a wide based on data allowed data triangulation at the analysis stage. When

One form of triangulation is member checking where participants are asked to confirm and share information which supplement video data.

As an example, in interview data a participant who expressed frustration in the moment about a form of disruptive practice of another participant, was able to place that frustration in context.

Limits of / problems with video data analysis

The choice of the videos to code more fully is rather skewed towards finding interesting data. Thus it may not be fully representative of the experience of all participants.

Choosing in a different way - around less interaction - would expose more barriers to participation to game making using the 3M model.

Only suitable pairs were chosen. This removes a lot of validity from any generalisation of claims that could be made.

Breaking of data into 5 min sections is practical but does not allow for some of the finer interactions and changes of behaviour to be accurately coded. There may be some value is coding some key interactions more carefully in more granular detail.

In summary, the coding process is more valuable as a tool to allow more qualitative reflection and analysis rather than a quantitative process as attempted above.

To make a quantitative approach work better X,Y and Z would need to be in place which was not suitable for this study because of A and B.

Limits of interview data_

The chief limit is the lack of repeated and regular interview data.

The reason are mostly practical, fitting in additional interviews with families withing a busy schedule.

It was depriorisited and less interviews were conducted than anticipated, and not as regularly.

This reduces factors of reliability as it reduces possibilities for

And prioritising more hands on work which was perceived as more benefitial and welcome by myself.

Discussion on evolving coding process - PERHAPS SHIFT TO LAST CHAPTER finally

The process of trying to decide a main focus was often tricky.

Take for example the process of Da in 2019-05-08 - where he was leading some fairly advanced work on researching and reading developer documentation and the constructing a new design pattern.

There are competing and overlapping intention and foci here for the participants.

Solve immediate problem - add new pattern
communicate cultural practice of finding, reading and adapting developer level documentation and help from forums.
give hands on experience to the notice coder

Most clearly there is an overlap here between the interpersonal working on a code problem collaboratively for educational purposes and the interactions with the wider cultural artefacts and practices from a professional environment.

Thus the process of writing this up was not neat. To overcome this issue…

In addition the personal plane was perhaps the most problematic to address given the data gathered. Even when participants were demonstrating their personal knowledge by sharing back or directly altering game code, this was as part of a social and cultural practice. Thus my interpretation involved ….

Generalisation of Research

Data Replicability

The elements of the design are online and included in appendices.

This study faces an issue of being culturally embodies, thus think descriptions of context are needed here [@hoadley_creating_2002].

The final chapter addresses the need to replicate this study in different contexts and the challenges involved in replicating the spontaneous emergent elements of specifics of this study.

As such chapter four can be conceptualised as a design narrative (see [@hoadley_creating_2002]) to address the common deficit of detail of the designed product and process. “the usual study presents a technology fully formed as if it had risen from the oceans like Venus herself”

” Would that all interventional research included this kind of rich description of the “treatment” so that one might infer whether the results were applicable elsewhere.”

Generalisation of Research

Generalisation is one of the guiding principles of DBR. While results are intertwined with context, designs should be assessed for generalisation to other settings.

Explore this in more detail - draw out a debate on how situated learning in this context - and wider debate is.

More than one iteration.
In different settings
Using different with different software

This is potentially complex to describe but worth it if wanting to talk about generalisation.

NOTE - This could be dropped or picked up again in the limitations section

The process of exploring this guided the development of the research questions.

Discussion on Methodology

Discussion / Conclusion

NOTE - WHAT KIND OF DISCUSSION WOULD BE NEEDED HERE? A RETURN TO POSSIBLE BARRIERS THAT THE METHODOLOGY AIMS TO OVER COME

Authenticity - creating value

IS THIS NOT COVERED IN CHAPTER 3 - UNDER JUSTIFICATION OF APPROACH / THEORETICAL FRAMEWORK?

Barab and colleagues discuss the importance of creating value in the world in their ecosystem approach [@barnett_ecosystem_2019]. This mirrors authenticity in applied social approaches - PBL as applied method of - situated learning ontology.

On the emergent and mutual nature of the design and research process

The experimental team consisted of Home Educating families . Learners acted as researcher participants to guide the next iteration of the game making program both directly and indirectly.

STAGE ONE Direct input was through requests and informal feedback and structured end-of-course interviews. Indirect input came from research data in the form on the games participants created, my research journal entries on my interactions with and observations of participants and recorded audio and video data of the participants and their computer screen capture.

On potential of involvement of parents

The presence of parents adds significant opportunities. I propose that this informal and exploratory setting with the participation of engaged parents, young people and their access to family learning dynamics and wider expertise provides potentially fertile ground in the generation of resources and domain practices. I explore the potential of this theme in more detail in the conclusion of this thesis.

How does this methodology relate to existing models of Change Laboratory & DBR and what does that add potentially

Keep this brief.

The smaller time frame of iterations, and more direct intervention in the ongoing / leading activity distiguishes this approach from a Change Laboratory. However, the analysis of tensions, and thus double stimulation to provoke evolutions of agency are alike.

The next chapter focuses in more depth of the mutual evolution of the learning design in a way which aligns with the design narative technique of DBR (hoadley? - find others!)

Chapter conclusion and link

In line the ethos of formative interventions and DBR this chapter has outlined a methodology which aknowledges the roles of my own input as a scholar/activist and that of participants. The process of working with participant input to the design of the learning intervention served several functions including: access to novel technical approaches proposed by parents and university volunteers; the surfacing of systemic conflicts hidden from my initial perception stemming from my pre-conceptions; observation of participant conflicts spurred the introduction of new tools to support development; and parental proficiency in adapting the learning environment to supporting their children leading to community-wide innovations in practice.

This chapter has described a hybrid approach based on the principles of the Helsinki school’s interpretation of formative intervention, supplemented by naturalistic research methods involving gathering data in-situ to find and describe emerging patterns of behaviour via thick decriptions and the more rapid iterations into the working practices common in DBR in the learning sciences and HCI approaches.

My role as a participant facilitator, designer and researcher presents some challenges which this chapter has addressed. While some decisions were to an extent intuative or reflexive based on my past experiences as a facilation of group productive processes involving technology, and for some this would present issues of validity, the process is in line with a TAS approach. While my auto-ethnographic notes are too extensive to effectively bring into the data analysis process, they chart the complex and iterative process of refining this reflection into learning and teaching resources. Several samples of journal entries are offered as evidence of this process in Appendix. 4.x

LINK

While the phases and overall structure of the learning design have been presented here, the following chapter, using the a technique of design narrative, explores the evolving areas of contradictions that drive these changes to the evolvling and expanding object of the group game making process.

Parked to replace or remove or rework

How Research Questions Evolved - JUSTIFICATION FOR CHANING FOCUS.

was driven by

Possible themes that had potential but which were disguarded included;

Computation thinking & systems thinking
Aligning with Project based learning

Themes emerging include the alternation of abstract and concrete as a suggested learning journey, explored in my journal. However this seems to be explored in research on semantic waves with studies of more breadth so it did not seem a good idea to follow that thread in my thesis.

There was something about the formalisation of this process into graph form which put me off. I also realised that the frameworks that I spent a while trying to make accessible and gameful in the design, grouping and theming of their presentation, had strong similarities to concepts of abstract curriculum if explored through a semantic waves lens. Curriculum seemed too well a well trod path for a PhD thesis.

Instead the community elements of the learning experience regained focus during the process of coding. There were particularly interesting elements of the learning environment which contributed to notable behaviours.

The area of in-depth study of learning and wider behaviours in the development of coding. Work on roles by Barron and developed by Roque showed promise identifying helpful roles and reflecting them back to participants. Such reflection sessions were not always replicable with a general home education group or other non-formal setting. Could this implemented in the design, in the way reflective techniques were implemented in to games. Thus making it more playful/ gameful.