SYSTEMS SAVVY: THEORY, MEASUREMENT, and IMPACT
•Télécharger en tant que PPTX, PDF•
1 j'aime•753 vues
Presentation at the 2014 Academy of Management Meetings. Systems savvy is the ability to grasp possible functions of technology tools and organizational processes and how these might be meshed to best effect. This is the academic work underlying the book, The Plugged-In Manager.
Recommandé
Recommandé
Contenu connexe
Similaire à SYSTEMS SAVVY: THEORY, MEASUREMENT, and IMPACT
Similaire à SYSTEMS SAVVY: THEORY, MEASUREMENT, and IMPACT (20)
Plus de Terri Griffith
Plus de Terri Griffith (20)
Dernier
Dernier (20)
SYSTEMS SAVVY: THEORY, MEASUREMENT, and IMPACT
- 1. SYSTEMS SAVVY: THEORY, MEASUREMENT, and IMPACT Terri Griffith, Santa Clara Univ John E. Sawyer, Univ of Delaware M. Scott Poole, Univ of Illinois Slideshare.net/TerriGriffith
- 2. #AOM2014_1836 Systems savvy is the ability to grasp possible functions/affordances of technology tools and organizational processes and how these might be meshed to best effect. Technology & Organizational Dimensions, with a touch of Emergence
- 3. #AOM2014_1836 AP Photo/Charles Rex Arbogast
- 5. #AOM2014_1836 “[a]lthough most studies up to this point have sufficed to simply show that social and the material are thoroughly intertwined, scholars are just beginning to consider how such intertwinement occurs” (Leonardi, 2012, p. 35).
- 6. #AOM2014_1836 We provide a model of systems savvy and a field validated measurement tool (Situational Judgment Task)
- 7. • Personal Innovativeness in IT – Agarwal & Prasad 1998 – Thatcher & Perrewe 2002 – Wang, Hsieh, Butler, & Hsu 2008 • Mann Gulch: Weick 1993 • Expertise – Johnson and Mervis 1997 – Murphy and Wright 1984 – Tanaka & Taylor 1991 • Practical Wisdom: Aristotle
- 8. Scenarios Interviews Outcome Ratings Focus Groups 6 Scaled Scenarios Experts Novices Field Validated Measure Sternberg, et al. 2000 Weekley,Ployhart, & Holtz 2006
- 10. #AOM2014_1836 Technology, Organization, & Emergence Related to research on the positive relationships between organizational change and tolerance for ambiguity and internal locus of control (e.g., Judge, Thoresen, Pucik, & Welbourne, 1999).
- 11. 0 0.5 1 1.5 2 2.5 3 3.5 4 toe to e avgT+avgO Social Media SysSavvy Novice Avg Ranking of Response by Sample
- 12. #AOM2014_1836 Comparison of SJT with Likert Scale Likert Example: When I adopt a new technology, I always consider other changes in my workflow that might help. E.g., MacDonald & Uncles (2007) consumer savvy scale
- 13. #AOM2014_1836 Binary Logistic Regression Results With Expert Status Regressed On SJT Score Model Summary -2 Log likelihood Cox & Snell R Square1 Nagelkerke R Square2 148.523a .229 .378 aChi-square=57.449, df=1, p<.001 Classification Tablea Predicted Percent Correctnovice savvy Observed novice 179 3 98.4 savvy 20 19 48.7 Overall Percent Correct Classification 89.6 a. The cut value is .500 SJT scale alone accurately classifies 89.6% of the respondents, while random assignment would differentiate only 50%.
- 14. #AOM2014_1836 Incremental Classification Accuracy of SJT Score Over Likert Scale Only Model Summary Step -2 Log likelihood Cox & Snell R Square Nagelkerke R Square 1 153.051a .213 .351 2 109.815a .353 .582 Likert Scale Classification Table Predicted Percent Correctnovice savvy Step 1 Observed novice 177 5 97.3 savvy 26 13 33.3 Overall Percent 86.0 SJT and Likert scale Classification Table Predicted Percentag e Correctnovice savvy Step 2 Observed novice 177 5 97.3 savvy 14 25 64.1 Overall Percent 91.4 SJT scale adds nothing to the correct classification of novices, but doubles the correct classification of system savvy individuals. Correct Classification of 91.4%
- 15. #AOM2014_1836 Future Research • Tool Use in Teams – Thomas & Bostrom 2010 – Majchrzak, Rice, Malhotra, King, & Ba, 2000 – Maznevski & Chudoba, 2000 – Montoya, Massey, & Lockwood, 2011 • Organizational Memory: Argote 2013 • Strategy -- IT Leveraging Competence – Pavlou & El Sawy 2006 • Technology Design – TSA, Healthcare.gov
- 18. Thank You Many Individuals Throughout
- 19. #AOM2014_1836 Preference Ordering of Scenario Responses Combined technology-organization-emergent (scenario responses offering intertwined human and technical dimensions with the acknowledgement of possible adjustments over time) Combined technology-organizations Emergent Organizations only Technology only 3 1 2 4.5
Notes de l'éditeur
- Intro John & Scott
- ..and to predict/avoid unintended consequences
- How many of you have been through this experience in the last week?
- Strong similarity for how you might be treated by the Philadelphia Police if you went out an robbed a liquor store tonight. My question is this, how much of a difference does it make to have your hands up or not? The original public pictures actually showed arms like this…. My claim is that the designers may not have had, or at least been able to act on, high levels of system savvy. There seemed to be a technocentric approach to the process in stead of a sociotechnical one. Most of the organizational and human changes have come far later, but not on the public table during the initial implementation. I look forward to all the decision making documents being made public so we can do an actual assessment.
- When we do address organizational and technological, we have less to say about how the intertwining happens. Leonardi, P. M. 2012. Materiality, Sociomateriality, and Socio-Technical Systems: What Do These Terms Mean? How are They Related? Do We Need Them?, Materiality and organizing: Social interaction in a technological world: 25-48: Oxford University Press.
- We believe that a field appropriate, validated tool will help us in studying the dynamics of acquiring system savvy and the dynamics of the intertwining process.
- Here is the version in the paper showing possible antecedents and how systems savvy can moderate the possible technical and organizational options. The model is built on: Agarwal, R., & Prasad, J. 1998. A Conceptual and Operational Definition of Personal Innovativeness in the Domain of Information Technology. Information Systems Research, 9(2): 204-215. Thatcher, J. B., & Perrewe, P. L. 2002. An empirical examination of individual traits as antecedents to computer anxiety and computer self-efficacy. MIS Quarterly, 26(4): 381-396 Wang, W., Hsieh, J. J. P. A., Butler, J. E., & Hsu, S. H. 2008. Innovate with complex information technologies: A theoretical model and empirical examination. Journal of Computer Information Systems, 49(1): 27-36. Weick, K. E. 1993. The Collapse of Sensemaking in Organizations: The Mann Gulch Disaster. Administrative Science Quarterly, 38: 628-652. Johnson, K. E., & Mervis, C. B. 1997. Effects of varying levels of expertise on the basic level of categorization. Journal of Experimental Psychology: General, 126(3): 248-277. Murphy, G. L., & Wright, J. C. 1984. Changes in conceptual structure with expertise: Differences between real-world experts and novices. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(1): 144. Tanaka, J. W., & Taylor, M. 1991. Object categories and expertise: Is the basic level in the eye of the beholder? Cognitive Psychology, 23(3): 457-482.
- I’ll save the details for the discussion, but we followed the techniques from Sternberg et al. & Weekley et al. to create and scale six situational judgment tasks and their responses. The prompt question is something that they couldn’t have learning in a book. More details. We identified and interviewed over 13 people who had been identified as systems savvy by their peers. Many of these stories appear in Griffith, T.L. (2011). The Plugged-In Manager: Get in Tune With Your People, Technology, and Organization to Thrive. Jossey-Bass. We used the techniques from Sternberg et al. & Weekley et al. to create scenario responses and then worked with two focus groups to refine those responses and create a new scenario. The next step was to create a scoring key using samples of 39 peer-identified systems savvy respondents and 182 novices. Weekley, J. A., Ployhart, R. E., & Holtz, B. C. (2006). On the Development of Situational Judgment Tests: Issues in Item Development, Scaling, and Scoring. In J. A. Weekley & R. E. Ployhart (Eds.), Situational Judgment Tests: Routledge. Sternberg, R. J., Forsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M., Snook, S. A., & Grigorenko, E. L. 2000. Practical Intelligence in Everyday Life. Cambridge: Cambridge University Press.
- Handout
- During the initial scenario collection we were expecting the interviewees to discuss sociotechnical answers – combinations of organization and technology. While they generally did this – they also would talk about letting events emerge. This brought us to Judge et al. Our final response items are what Weekley, et al., (2006, p.166) call construct-based response options. Judge, T. A., Thoresen, C. J., Pucik, V., & Welbourne, T. M. 1999. Managerial coping with organizational change: A dispositional perspective. Journal of Applied Psychology, 84(1): 107-122.
- Average Ranking of Systems Savvy sample versus Novice Sample. Note that the systems savvy sample has a preference for the Technology-Organization-Emergent response, and a disinclination to the unilateral responses using just technology or just and organizational outcome. The Novices are much less differentiated. We started out with six scenarios. One was unable to differentiate between novices and systems savvy – so validated set includes only five scenarios.
- The important story about the SJT scales is that it helps us differentiate savvy people from novices. Slide shows that the SJT scale alone accurately classifies 89.6% of the respondents, while random assignment would differentiate only 50%.
- Slide shows that the SJT scale alone accurately classifies 89.6% of the respondents, while random assignment would differentiate only 50%. Slide shows that the likert scaled measure correctly classifies novices, and that the SJT scale does not improve the differentiation of novices, but doubles the accuracy of classification of savvy individuals.
- Thomas and Bostrom (2010), for example, provide a new lens for exogenous elements that may trigger improvements in how tools are used by teams. The value of organizational memory is limited by how well the organization’s members know how to store and access memories, and/or how well that knowledge is placed (for example, in organizational routines or hardware architectures) for more tacit use. Thomas, D., & Bostrom, R. P. 2010. Vital signs for virtual teams: An empirically developed trigger model for technology adaptation interventions. MIS Quarterly, 34(1): 115-142. Majchrzak, A., Rice, R. E., Malhotra, A., King, N., & Ba, S. 2000. Technology Adaptation: The Case of a Computer-Supported Inter-Organizational Virtual Team. Management Information Systems Quarterly, 24(4): 569-600. Maznevski, M. L., & Chudoba, K. M. 2000. Bridging Space Over Time: Global Virtual Team Dynamics and Effectiveness. Organization Science, 11(5): 473-492. Montoya, M. M., Massey, A. P., & Lockwood, N. S. 2011. 3D Collaborative Virtual Environments: Exploring the Link between Collaborative Behaviors and Team Performance. Decision Sciences, 42(2): 451-476. Argote, L. 2013. Organizational Memory, Organizational Learning: 85-113: Springer. Pavlou, P. A., & El Sawy, O. A. 2006. From IT leveraging competence to competitive advantage in turbulent environments: The case of new product development. Information Systems Research, 17(3): 198-227.
- Our systems continue to get bigger, more complex, and more critical to day to day action. Systems savvy and its measurement are expected to provide insights into the design and implementation of complex, real-world systems. http://www.zdnet.com/government-admits-to-design-flaws-in-healthcare-gov-7000021600/ http://online.wsj.com/news/articles/SB10001424052702304441404579119740283413018