From experience to knowledge in professional IT management education: exploring the applicability of classroom learning to real-life contexts

Thus, if knowing (rather than knowledge) is a capability that emerges from, is embodied by, and embedded in recurrent social practices (Foucault, 1980), then designing training materials for classroom-based professional education presents a particular challenge (Becker, 1972). Not only is it necessary to account for the epistemological and ethical entailments of knowledge sharing (Duguid, 2005) between a trainer and participants from a potentially vast array of industrial sectors (Brown and Duguid, 1991, 2001), but also for the abstraction of the practitioner from the locus of their practice.

A lot of the practice-based research that explore how practitioners develop and demonstrate their expertise focuses on the learning that occurs whilst they are engaged in their work, for example Lave and Wenger’s (1991) studies of the apprenticeship models of learning amongst butchers and seamstresses; peer learning and community development between insurance brokers (Wenger, 1998); or the negotiation, maintenance, and reinvention of medical professionals’ identity (Korica and Molloy, 2010). Another key theme examines the formation of identity and cohesion of communities in virtual or online communities (c.f. Mills et al., 2014; Oztok, 2013). There is a thread of commonality running through these studies that transcends the variety of situations examined, however. Knowing is demonstrated and learning takes place through the learner/practitioners’ active engagement within the community: physically (with practices and tools), mentally (reflection on actions and outcomes, storytelling), and emotionally (through connections and interconnections).

Our key aim in this research was to explore whether it is possible for the classroom setting to approximate real-life work contexts, thereby enabling the active physical, mental, and emotional engagement of learner/practitioners with their community of practice, which have been demonstrated in the literature to be central to learning. In what ways is this accomplished, and/or how might training programmes be adapted to compensate for the abstraction of the learners from the locus of their daily practice? By understanding better how to establish this dynamic in the classroom, we believe that we can provide a learning experience that is more readily applicable to real-life contexts and thus, enable more meaningful interchange of ideas and more robust learning.

The IT-CMF is a framework designed to manage information technology (IT) to deliver business value across four main vectors, or macro-capabilities: Managing the IT Budget, which comprises the input to the production process; Managing the IT Capability, which is seen as the production engine; Managing IT for Business Value, which is the core output; and Managing IT like a Business, which provides feedback to adjust the inputs to optimise the output value (IVI, 2014c).

Each macro-capability is further broken down into distinct yet interrelated critical capabilities – currently 35 in total – that outline the key activities and procedures that will enable the IT organisation to plan and deliver IT solutions, measure the business value of its initiatives, and understand where IT is positioned within the wider organisational context. Each critical capability is described in terms of capability-building blocks, or elements that should be addressed at each level of maturity; a summary description of each level of maturity from initial (unmanaged) through to optimising (value-centric IT management); and a roadmap for incrementally improving maturity in the key areas identified (IVI, 2014c). Higher levels of maturity are achieved and evidenced through improved alignment between IT and the rest of the business, thereby enhancing both the value of IT to the business and the value the business can generate from its investments in IT.

Unlike other maturity models, for example, CMMi or Six Sigma, the IT-CMF does not necessarily recommend an organisation should aim for the highest levels of maturity across all critical capabilities. Rather, by taking an integrated and holistic approach to IT management within its organisational, industrial, and national contexts, the IT-CMF helps individual organisations to identify the areas of critical importance to its strategic and operational success, and provides an incremental roadmap to help improve IT capabilities to a contextually appropriate level in those key areas.

Within IVI, the Education team has developed a five-tier training programme, and it is envisaged that professionals will move through these tiers as they develop and deepen their practice. Initially, participants develop a basic knowledge and elaborate their understanding of the IT-CMF through Tier 1 and Tier 2 courses. In Tier 3, 4 and 5 courses, however, participants begin to apply and evaluate their learning, becoming more reflexive and critical consumers of the IT-CMF and transforming their IT management skills in a way that reflects the intertwining of their prior knowledge with their experience of using the IT-CMF in real-world contexts.

Tier 1, a one-day ‘passport’ course, offers an introduction to the IT-CMF aimed at CIOs and high-level IT managers. By the end of the training, participants can articulate the business value of the framework, describe its structure, and explain how the framework can be used to assess the challenges faced by their organisation. Since assessment is an intrinsic part of using IT-CMF, they will also learn about the types and application of IT-CMF assessments, and the processes used to carry out an assessment. This is achieved with a combination of didactic presentation, hands-on exercises, and facilitated class discussion.

Tier 2 is a three-day offering (the Passport course forms the first day of this level of training) geared toward managerial level IT practitioners and IT consultants. Building on the learning of the Passport course, participants further develop their understanding of how to navigate the components of IT-CMF critical capabilities, and demonstrate how particular critical capabilities can be used to improve the business value of IT. This course focuses on two or three specific critical capabilities, such that participants can deepen their knowledge of these capabilities and apply this understanding to increase IT capability maturity. As before, this is achieved with a combination of didactic presentation, hands-on exercises, and facilitated class discussion.

Tier 3 and 4 training courses are intended to certify participants as assessors and change management leaders using the IT-CMF. Finally, training at Tier 5 has been encompassed by the MSc in IT Management offered through the IVI by the School of Business at MU.

The collection of primary data was conducted over two separate sessions between April and June 2012. Participants on two of IVI’s three-day Tier 2 training courses were invited to take part in a brief focus group at the end of both the first and the third days’ training. Sixteen contributed to the focus group at the end of Session 1/Day 1, of whom thirteen participated at the end of Session 1/Day 3. There were seven contributors to both of the Session 2 focus groups.

Conscious that our participants had just completed a full day’s training programme, the focus group sessions lasted no more than thirty minutes and aimed to understand how the exercises that are built into the training materials were used by the course participants to make sense of the IT-CMF within their own work-life contexts. The timing of these focus groups was deliberate to capture participants’ views at different stages of their learning journey. The information gathered at this time was also fed back to the course designers.

Each participant was subsequently invited to participate in an interview with the authors. These interviews, which lasted about thirty minutes, were purposely scheduled to take place around two weeks after the training. This was to allow participants the chance to utilise the IT-CMF in their work, giving time for the framework to be further contextualised and grounded within their own practice, while the training programme was still a fairly recent memory. The data gathered thus far represent preliminary findings, and it is intended to carry on with this research agenda to confirm and expand the results to continue to improve the learning environment that we provide to our course participants.

A great deal of complex information is presented to participants in the three-day course, and it is easy for participants to become overwhelmed or fatigued by this volume, depth and rate. The exercises have been designed to offer a change of pace: allowing participants to slow down; read, understand, and absorb the materials; and try to apply them in a non-pressurised context. These exercises are usually conducted as group work, with between two and four members. Quite often, the groups will form naturally, with participants from the same ‘home’ organisation teaming up together.

From an ‘application to real-life’ perspective, there are two types of exercise: Type 1 calls on participants to use their own real-world knowledge and experience and apply an aspect of the IT-CMF to develop a solution to a real-life issue; while Type 2 presents participants with a previously constructed problem, for which they must develop a solution. For the purposes of this research, we focused on two specific exercises, one of each type.

In the first exercise (Type 1), which we refer to as the Clustering exercise, participants are presented with a fairly large selection of pre-defined clusters of critical capabilities aimed at solving particular types of commonly reported problems or issues. Participants must first choose a real-life problem that they wish to tackle and then identify with which of the listed issues this resonates most closely. They must then narrow down the cluster of critical capabilities to the three key capabilities they will choose to address the problem. This exercise is on the first day’s programme.

The second exercise we examined (Type 2) is conducted on the afternoon of the third day’s training. Here, participants are presented with a pre-prepared problem, framed as an IT-CMF assessment report, from which they must identify the assessed organisation’s problem(s) around a specified critical capability. Using this report and the POMs

The POMs (Practices – Outcomes – Metrics) are one of the artefacts associated with each of the critical capabilities that outline indicative practices, outcomes, and metrics at each level of maturity and correspond to each of the capability building blocks (CBBs). These POMs are not a prescriptive tool, but rather offer some guidelines to assessors (and assessed organisations) as to the types of behaviours and practices for which they should be seeking evidence, or aspiring towards.

Generic exercises allow those who are new to the IT-CMF to learn about how and when it can be a useful tool, giving them a chance to learn ‘the rules’ (Dreyfus and Dreyfus, 2005), that is, when it might be appropriate to use the tool and which of its permutations is more appropriate for the particular circumstances described. We found that having the participants use examples from their own experiences in the earlier exercises helped them to ground what they were learning within the context of their own experiential domain. By effectively acting as a thought experiment, the participants engaged with the exercises in a highly individualised and practical manner.

Through their engagement with the exercises, the participants came to view the IT-CMF as a flexible tool that can be used to address a wide range of specific issues.

Rather than being a series of stand-alone critical capabilities, it is an intricate and interlinked framework that can comprehensively address real-life issues.

Engaging with the exercises gave participants the opportunity to test the framework against a scenario with which they were intimately familiar.

Participants thus began to identify for themselves how the IT-CMF might have been applied in a real-life situation.

As IT practitioners and managers, the participants already had first-hand experience of using various other tools and frameworks to guide their work. Whilst they were still learning about the intricacies of this particular tool, their fluency with applying such frameworks permitted an accelerated move away from rote-learning models and toward a more intuitive and competent application of the IT-CMF to their identified organisational issues based on their own expertise (Dreyfus and Dreyfus, 2005).

A relevant practice within a domain of experience is considered to be any activity ‘that the action of engaging in it is consequential for the development of the activity’ (Feldman and Orlikowski, 2011: 42). Physically engaging in practical activity is, therefore, just one way in which IT practitioners ‘do’ their work. Similarly, thought experiments, thinking through problems (both real and hypothetical), and discussing hypothetical solutions are also ways in which the activity of IT practitioners plays out. The learning environment we tried to create emulated these thought experiments and encouraged participants to (hypothetically) apply the tools they were learning about within the context their own real-life experience.

Despite the variation in participants’ home organisation, including factors such as the size, sector, focus, and even country of operation, the examples discussed were, to a greater or lesser extent, analogous and relevant to all the participants.

At the very least, the issues of concern were understood by all, emphasising that as IT professionals, their community of practice was capable of transcending the apparent specificities of their organisational bases. This eased some of our concerns in terms of trying to develop materials of relevance to IT practitioners from a range of backgrounds.

Unexpectedly, we found that the training programmes also served as a sales tool, both for the IT-CMF itself and for the participants taking the framework back to their organisations and/or clients. On the one hand, some participants reported that they were evaluating a number of potential IT management frameworks and had attended the training to learn more about the IT-CMF. Through engaging with the practical exercises, they not only developed an understanding of the IT-CMF, but also realised that this was a framework that would work with their clients.

Through the training programme, participants began to more clearly identify and articulate how they might leverage various aspects of the framework to sell it, either within their own organisation, or to client organisations.

Another participant saw engaging with the IT-CMF and the certification process through the tiered education programme, as being a useful way to move up the career ladder. By approaching the IT organisation from a managerial perspective, and understanding what and how IT can contribute to developing and enhancing the value of IT to the overall business, he felt that he was strengthening his position in terms of career progression.

Participants thus identified the potential to enhance their professional identity and legitimacy through demonstrating fluency or expertise with the IT-CMF – not simply with their learning peers, but more significantly within their home organisations and wider networks (Dreyfus and Dreyfus, 2005; Korica and Molloy, 2010).

The views expressed by the participants in the focus group sessions also helped us to understand how aspects of the course delivery were perceived. Based on this, we were able to support some elements of the programme design and make some adjustments to other aspects.

Participants were each supplied with a professionally bound reference book that listed all the critical capabilities and included a high-level description and an overview of the capability-building blocks for each. This document was found to be difficult to use, particularly during the Type 2 exercise, which called for much cross-referencing and ‘riffling’ through the pages. The tight binding also caused frustration as the books were liable to close over and participants complained of ‘losing their page.’ Based on the suggestions from both sets of participants, this book has since been republished: within each macro-capability, the critical capabilities are now listed alphabetically; page numbers are provided for each; and the book is now spiral bound to enable easier access to each page.

There were different course presenters for each of the training sessions explored. The differences in their presentation styles and approaches to facilitating the group discussions could have a considerable effect on how the course participants engaged with the exercises, and how they subsequently described their learning experiences. In carrying on with our research in this area, we would intend to explore this potential correlation further.

Similarly, there were a different set of participants for each of the training sessions, with a different mix of genders, nationalities, sectors, organisation sizes, and types represented. Whilst both groups were energetic and motivated, the inter-group dynamics were markedly different, and this also had an impact on the direction of the focus group discussions.

Rather than trying to ‘control’ for these differences between presenters and participant groups, we acknowledge and welcome the disparities, which allow ‘room for the complexities and contradictions’ (Korica and Molloy, 2010: 1882) to emerge. As we expand this research agenda, we would intend to harness these differences to highlight various aspects of the learning experience, thereby improving the training programmes delivered.

Although this research represents preliminary findings based on a limited sample, we believe that our findings are of significance to the design and development of IT-CMF training programmes and can, to some extent, be extrapolated to IT professional education in general.