Principles for Effective Data Visualization and Presentation
•
1 j'aime•398 vues
This is my Avoiding Chartjunk and Slideuments presentation from CES 3013. Thanks to all attending! John Burrett
Recommandé
Contenu connexe
Dernier
Dernier (20)
En vedette
En vedette (20)
Principles for Effective Data Visualization and Presentation
- 1. Avoiding “chartjunk” and “slideuments” Principles for visual presentation of data and findings
- 6. 70% Iconic memory Working memory Preattentive processing “Chunks” of memory
- 20. Sales (US $) North America Europe South America Asia Central America Middle East
- 21. 0 50000 100000 150000 200000 250000 300000 350000 400000 450000 500000 Sales (US $) 0 200 400 600 North America Europe South America Asia Central America Middle East Thousands Sales (US $) Sales (US $) North America 489,384 Europe 273,854 South America 243,998 Asia 134,847 Central America 128,054 Middle East 75,987 Africa 34,856 1,380,980 Sales (US $) North America South America Central America Europe Asia Middle East Africa
- 23. Real Time Visual Analytics Talking face to face with your data
- 37. D
- 42. Slideuments • Slideuments are documents viewed with presentation software – If you have 75 words you have a document – If you have 50 words, your slide deck is a teleprompter • Presentations should focus on the presenter and her ideas and concepts – Slides should reinforce the content visually rather than create distraction – This allows the audience to focus comfortably on both
- 44. 30% use parks yearly 25% occasionally People already love the parks system
- 45. People already love the parks system 30% use parks yearly 25% occasionally
- 48. - Here is a sample slide - I made up the statistics - But the point is how the slide supports the message Your slides Notes for you only Handouts for later
- 51. 0% 5% 10% 15% 20% 25% 30% 35% Don't Know Never Seldom Occasionally Yearly Park Use 2011
- 52. Simple Works! • Succinct text • Crisp thoughts • Big ideas • Clear mnemonic • Relieved audience
- 57. Am I really going to do all of that???
- 58. A creative process ….
- 62. PRINCIPLES
Notes de l'éditeur
- REALLY PLEASED TO HAVE THIS OPPORTUNITY – A COUPLE OF THINGSMENTION LACK OF TRANSLATIONS AND WHYASK TO HOLD QUESTIONS TILL ENDWho is here:Because they are interested in better presenting their data and findings?Because they would like to see better presentation of data and findings?Because they are curious?We have all been in situations were we see that the importance of our data and findings is not getting acrossMight be as we are discussing a report around a boardroom tableOr presenting findings in to a conference audienceWhere you think the really interesting points are, you don’t really see a recognition of that,,,, or you find yourself exactingly describing your way through the data with the overall feeling that you are just “losing them”….I have been in and around evaluation and policy research and development for a long time, and even political research and communications for a while. I’ve learned a lot about communicating,but still never really saw what I thought were foundational principles that could be applied to everyday work.Until I started poking around in the field of data visualization, and saw that a sub-set of that world were talking mainly about clarity, rather than flash. And the path to clarity was based on principles… And it turned out that there were principles for presenting data, and some overlapping and related principles for making verbal/visual presentations. It wasn’t just a matter of personal taste and touch.So that’s what this is about today – THIS WILL BE AN INTRODUCTION TO SOME IMPORTANT CONCEPTS – BUT ONLY AND INTRODUCTION – THERE’S LOTS TO KNOW.The BIG IDEA – my goal – is that you will be convinced that you might be able to increase the impact and utility of your work if you learn about and use some of these principles for presenting data and findings.
- Program Evaluators work to very high standards of scientific method and rigour. You, we, take pride in that, and insist upon that, and rightly so. That dedication to the truth is key to the role that evaluation can play in our organizations and the larger world. But we often fail to influence, even armed with our truth. Why?Yes, part of it is that the truth can be unpalatable, and policy imperatives sometimes displace dispassionate analysis,,,, BUTPart of it is that we, as evaluators, just like others, are not necessarily very well equipped to communicate what we need to communicate in order to be heard. Our communicating isn’t up to the same standards as our analysis.Its not our fault – economists, statisticians, social scientist, etcetc are not generally taught this stuff. Its supposed to be “natural” or something,,What if there were principles to follow, and presentation wasn’t just some artsy / marketing netherworld?
- Chartjunk and slideuments are everywhere. Here you see some chart junk. I know most of you wouldn’t do this,,, but lots do,,, and I have……Chart junk is the stuff that doesn’t help, or in fact detracts, from a chart or graph’s ability to quickly and clearly impart information. Without all of the junk,, this could be a very simple graphic showing three numbers and a comparative relationship instantly. But as it is, with the the 3-D, grid lines, faux marble, etc,, who can tell me much very clearly about what this says?We’ll do deconstruction of chart junk in a bit…
- Here’s a familiar one. Maybe 3-D will help!!!Not really, huh? But we see this all the time. So this is a problem. And the problem is foisted on us, considerably, by the folks who make our data analysis and presentation software. This pie would take no effort at all to produce in Excel,, and in fact is one of the less objectionable options in Excel’s charts menu.
- Fortunately, some of the big brains in and around the hotbeds of innovation that give us the computing equipment with which we can inflict such horrors have turned to how make the computers work for us…… Key examples woud be Stephen Few, USC Berkeley, and Edward Tufte (Yale), who have advanced the subject of the visual presentation of data by leaps and boundsAdditionally, Nancy Duarte, MountainViewCalif (located cheek to jowl with Google HQ) and Garr Reynolds, formerly of Apple Inc and now teaching in Japan, promote the keys to impactful verbal/visual presentation. Their client list is a whose who of big business and entertainmentI took some training both with Duarte Design and with Stephen Few this summer, and it was eye opening.
- We are visual creatures.Vision, of all of our senses, is the most powerful and efficient channel for receiving information from the world around us.Did you know that 70 % of of the sense receptors in our bodies are dedicated to vision?And what we do with that visual information is profound. One study, from the the US Dept of Labour, 1996, says that 83% of what we retain from something like a presentation occurs visually.There is, of course, a lot that you can know about the science and mechanics of seeing and perceiving. This will just be a scratch on the surface, but its what we need to know.. We know that our eyes basically work like cameras. And the machine metaphor, using computers, is helpful when describing the memory functions of the brain. There are 3 types of memory that process visual information in our brainsIconic memory – like the keyboard buffer memory of your computer, there just until there is room for fresh inputWorking memory – like RAM – very high speed processing, interpreting and responseLong term memory, much like a hard drive conceptually, and not of as much concern here. Information passes from our eyes into our Iconic Memory, where extremely rapid processing takes place before the information goes to the Working Memory.This part of perceptual processing is automatic and unconscious, and is thus called pre-attentive processing, as opposed to the higher level, conscious, attentive, processing occurring in the working memory.Preattentive processing is an extremely fast process of recognition, and nothing more. Attentive processing, is sequential, takes more time, and results in learning, understanding and memory.Preattentive processing detects certain attributes, such as colour and the location of objects in 2-D space. And this means that understanding those attributes is very useful in helping your audience see what you need them to see, immediately.Working memory is short term, and it has limited storage capacity.Only a few “chunks” or discrete pieces of information, can be stored and used in working memory at a time. So, again, it is important to understand this, and it has important implications. For example, if you do a graph with a long list of labels in the legend, you will force the viewer to go back and forth to refresh those “chunks”
- Here’s an example of the difference in speed of attentive and preattentive processing. I’ll give you a few seconds to tell me how many fives are in a block of numbers. Don’t shout out your answer. GONow let’s try it using your preattentive processing. GOThis time, the preattentive attribute of colour intensity made it easy. The attentive processing was serial, and much slower. So if you want you audience to see what you mean, rather than sit there working on it, you know what to do.
- Here are the preattentive attributes of visual processing identified by Stephen Few as most relevant to static tables and graphs (there are others, as identified by Colin Ware, such as motion). These will all instantly make a piece of information pop out in our minds.Only two are very good for encoding quantitative information (that is – we percieve them as having quantitative meaning):Length of a line2-D positionand less so:Width of a lineIntensity of hueSize of a shapeSo how you use them matters, but they are useful.
- This is why bar graphs, line graphs and scatter plots are the most effective graphical devices for quantitative information – length of line (bar graph), and 2-D space (line graph and scatter plot).It’s also why other representations are much less effective. Consider the use of the 2-D areas of objects to represent the relative quantitative magnitudes of measures of something, which we see all of the time, particularly in newspapers. (PUSH)
- How much bigger is the large circle than the small circle? If the small circle has a value of 1, what is the value of the large circle?The answer is 16. That’s just part of the problem with pie charts, but its enough. Whenever you can, use length or 2-D position to encode quantitative information in graphs.Similar demonstrations exist for the other attributes, like colour, but we don’t have time.
- So, there are lots of useful implications of this knowledge. Besides the use of preattentive attributes of visual perception to make your message stand out and properly encode quantitative information, how about the choice between tables and charts?This involves the characteristics of our working memories. PUSH
- Tables are best when you are concerned with the precise nature of individual values. Tables let you look it up. This is attentive processing, and is done chunk by chunk. Graphs show entire series of values at once, revealing patterns, trends and exceptions.
- You saw much of this preattentively. That’s what graphs are best at.All of the advantages of graphs over tables are multiplied as the number of data elements grows.What do we see, by the way?The following is a great example of this choice in function between tables and graphs. (PUSH)
- Here is a table of job satisfaction index numbers, broken into a cross tabulation of 2 income classes by two college degree and age classes. Quickly, does anyone see the one relationship that really jumps out as unique?PUSHNow? Preattentive – 2-D position (and colour for good measure)
- Another important application of the principle that our working memories, without possibility of an upgrade, can hold only so much information at a time, is the data to ink ratio.
- SHOW WHAT IS DATA INK AND WHAT IS NOTData ink is that ink that changes if the data changes. The rest is non-data ink. Some non-data ink is required to make sense of a graph, but the general principle, so as not to overtax our processors, is (PUSH)
- Here’s a simple exampleGrid lines Heaviness of the axis lines Could also carp with the tick marks and units of measure, but that’s where it starts to slide into art, more than science.
- Here’s another great data-ink ratio example. How often have you seen this?
- So let’s do a little example. Its more fun to do a detailed bit-by-bit deconstruction and reconstruction of a graphic using our principles,, but here’s a short form example. Here is a small set of data that I made up,, copied from Excel PUSH - within 10 seconds Excel will do this,,,,,, but now we know that our brains are having trouble with lots of this – 3-D representation, relative areas of pieces of a round object, non-data ink, and an agenda that is giving our working memory trouble.ONE thing to say, though, is that the colours are not too bad – at least they aren’t primary or bright colours. What’s the matter with primary colours? (PUSH)Because after the age of about two, bright and primary coloursdo not enhance our ability to distinguish objects from one another, and they then just become annoying and hard to look at. PUSH
- Taking out the 3-D effect helps, but it is still difficult to get a feel for the relative sales volumes in the various countries. The pie slices ask us to use curving lines and angles to see the data. Moreover, the legend completely defeats your working memory – too many chunks. Let’s abandon the pie chart,,, but while the bar chart is better, again, the 3-D representation that Excel loves to give us doesn’t help, and neither do the heavy grid lines, long numbers on the Y axis and slanted labels on the x axis. (PUSH) How about something a lot more calm, using preattentive attributes and little non-data ink? Or if we preferred the precision of a table, how about something like this (again with preattentive processing working for us).?The bar chart and table are almost interchangeable here. But all of the advantages of graphs over charts are multiplied as the number of data elements grows.
- WHY THIS MATTERS - YOU DON’T HAVE LONG TO MAKE YOUR POINTYou don’t want your audience to have to WORK at seeing what they need to see – you want them to see it immediately and then listen to you explaining it. This helps you whenever you present your data, in a report, or as part of a verbal report. THE MORE IMMEDIATE YOU NEED APREHENSION OF YOUR DATA TO BE, such as in a briefing on key findings, the more important this is. Use of principles of visual perception and memory are also very important for DASHBOARD DESIGN– where the whole point is immediate apprehension of what you need to knowOK, WE’VE TALKED ABOUT DESIGNING DATA DISPLAYS,,, NOW LET’S TALK ABOUT PRESENTATIONS
- Great for exploring your data – you can make patterns, exceptions, etc jump out, without engaging in a lot of exploratory data crunching – I’ll give you a demo on that sometime if you like… but here are some examples..I just pulled out a few things that I thought would be interesting for this demonstration of the power of visual data analysis
- Visual data analysis, using what we know now about how to display data effectively, can do most of the work we need to do with data. Not always but in most cases. GET STEPHEN FEW QUOTE Here’s a first look at the data, analysed live. This took virtually no time whatsoever, using software capable of instant data visualization. So let’s dig a little deeper – we want to see where these sales are coming from and why some are so high, or low……(the data graphics in this presentation were produced with Tableau Desktop™)
- Again, dragging and dropping in a new variable, product type, we can see that the picture is not so simple … the primary sources of sales vary across the states… it’s a bit messy, so let’s sort on one of the categories….
- There we go, sorted instantly. Now we see more clearly some of the key differences in the sales of the product types across the states by simply looking at the shapes of the distributions relative to the sorted one and each other.This is a visual application of analysis of variance.
- With careful application of visual design principles, standard presentations like scatterplots can be very informative. Here’s a display of profits as related to sales, and where the problems are….
- Visual analysis then lets us go to displays like this example of “small multiples”, that present a lot of information at once. Ok, THAT’s PRESENTING DATA ----- HOW ABOUT PRESENTATIONS?
- I was getting really tired of terrible, boring presentations, both seeing them and giving them… there must be a better way…..
- How is it that some people are able to rivet us with their presentation skills? There had to be an answer.
- So I went to see what the masters of the verbal-visual presentation form did, and boy, was it eye-opening! Duarte Design, Mountain View California…
- Darth didn’t win this. He failed badly in terms of finding common purpose and empathetic connection with his audience. But he DID make himself and his message very clear, and he had Luke’s attention. In doing so, what did Darth NOT do?
- Right. He didn’t use a slideument. And you shouldn’t either,, if you want your audience’s attention. We are smart, professional people. We have interesting information to relay, that could really benefit our audiences.
- We want our audiences to look like the people on the LEFT….. Someone has gotten their ATTENTIONHow come they so often look like the people on the RIGHT?Chances are these people were directed by the photographer to imagine they were watching yet another PowerPoint presentation. Again, it’s NOT REALLY OUR FAULT,,, our educations have not usually including learning how to communicate with a presentation, but….AUDIECES in government AND business routinely GET BORED TO DEATH, because we (I certainly have) commit the 6 DEADLY SINS OF PRESENTATION
- I HAVE COMMITTEDall of these, while feeling very proud of what a good presenter I was. And I bet most of you have too.WE DON’T HAVE TIME for all of the deadly sins today, so let’s focus on my FAVOURITE, and I think most FUNDAMENTALl of them: BULLETS, SUB-BULLETS AND SLIDUMENTSWHATS A SLIDEUMENT?
- READ IT OUT LOUDWhat did we just do? What we just did is called a “’READ ALONG”Chances are pretty much all of you were kind of HALF LISTENING to me, while you read the slide. In fact, you probably READ THROUGH IT AHEAD OF ME.You could have probably done better and be less annoyed if I hadn’t read the slide out loud. But I did, and people do this all the time.That’s what a slideument IS.WHY IS THIS SO BAD? Because people will EITHER LISTEN to a presenter OR READ the presentation. They will not do both. People tend to focus on one STREAM OF VERBAL COMMUNICATION at a time – listening and reading are conflicting activities. On the other hand, ITS NATURAL for people to simultaneously pay attention to both VERBAL AND VISUAL COMMUNICATION – so slides can serve as a VISUAL AID that reinforces the speaker’s messageSo what are we SUPPOSED to do with Slides??
- I am going to differentiate here, a bit, between presenting to executives, your boss etc,,, and speaking in more of a conference or public setting…. But the key points are going to hold for both types of presentation settings.. Let’s look at doing a presentation for a larger crowd, with a defined presentation time slot.
- FOR SOMETHING LIKE A CONFERENCE, ETC WHERE YOU ARE CONVEYING IDEAS, YOU MIGHT USE A FORMAT LIKE THISWHEN YOU DECIDED TO DO A PRESENTATION, instead of writing and circulating a technical document, you chose a form of communication that is best for ENGAGEMENT, MOTIVATION AND ACTIVATION. IF ALL YOU WANT TO DO IS TRANSMIT FACTS AND FIGURES , and you are not trying to make any point at all, including such points as “this information requires further scrutiny,,, or action…”, then consider not doing a presentation.BUT you’ve decided to do a presentation. SO TO DO IT WELL , you need to use slides as visual support for the ideas that you want to communicate and the points that you want to make.Here’s a not-too-bad slide for that kind of purpose. SUPPOSE YOU WERE WORKING FOR PARKS CANADA, and you have found that, while parks are reasonably well utilized, there was a subset of occasional customers, who, if they could be swayed , could up utilization considerably.YOU DON’T NEED the slide to say all of that. YOU need to say all of that, in order to be persuasive. THE SLIDE REINFORCES the idea of how nice the parks are and how people should want to use them…. And the slide is a place, if you want, to put a couple of key numbers or words to reinforce your message.
- THE INGREDIENTS OF A GREAT SLIDE – (click the words in)BACKGROUNDThis is the container for your ideasCan be anything or nothingReject default templates and default slide junk – use what will help present your idea, reflect your intent, or brandBut don’t make the background a busy work of “art” itself – it should not compete – a background is like real estate… build wiselyCOLOUR Colour is crucial – sets a tone and helps establish for the audience what to expect Pick colours considering – Who is your audience, What industry are you in, Who are you? Learn about concepts like “the colour wheel” that helps choose a harmonious paletteTEXT - 3 SECOND RULE – presentations are a “glance media”, like billboards – ASK YOURSELF - can your message be processed effectively within three seconds? - QUICK PROCESSING is aided by GOOD TYPESETTING – different fonts have different personalities – but restrict yourself to TWO FONTS PER PRESENTATION – one for headlines, titles, and the other for text blocks - HOW MANY WORD SHOULD BE ON A SLIDE? – there is no rule – use what you need to present the idea and to act as a mnemonic if you like, but go for a VERY LOW WORD COUNT, and try to minimise BULLETS – KEEP THEM FEW AND SPARSEIMAGESNot absolutely necessary but…..Help reinforce and reflect the message, its context, and the audience – industry, customer experience, real-life situations Avoid cheesy images and clip artUse professional photographs and images – or your own if you canBuild a libraryAND CONSISTENCY – use the same styles of these elements throughout your presentation (and can use the odd inconsistency to stand out)ANOTHER EXAMPLE WITH A BIT LESS GOING ON AND MORE “WHITE SPACE”
- This slide,, IS A GOOD EXAMPLE OF KEY PRINCIPLES OF SLIDE DESIGN:ARRANGEMENT – here we have good CONTRAST, use of SPACE, and FLOW (reading left to right, 4 quadrants of “REAL ESTATE”, and the RULE OF THIRDS)VISUAL ELEMENTS – BACKGROUND,,, TEXT AND IMAGES. Note also the use of FULL BLEED slidesMOVEMENT – not much movement here,,, although the DIRECTION OF ELEMENTS relative to the text and attribution are significant.
- OK, but it HELPS ME TO HAVE MY POINTS IN FRONT of me. How am I going to do without that?And my AUDIENCE WANTS TO SEE THE DETAILS– they insist that ALL of the information be there – they want the whole DECK.
- IF YOU ARE DOING A PRESENTATION, you cannot ask your slides to do it all. Use your SLIDES as they are meant to be used, use your NOTES, seen by no one else, to remind you of what you have REHEARSED, and IF YOU MUST, use a HANDOUT,, to be handed out AFTER the presentation, preferably. Your slides should be completely INADEQUATE AS A HANDOUTYou can deviate from all of this if you have a particular reason - Humour, in this case.
- LET’S DIGRESS FOR A MOMONT ON PRESENTING TO BUSY EXECUTIVES OR MANAGEMENTTHING YOU ALEADY KNOW…- Keep SHORT – say it will be short- SUMMARIZE the key messages first – so they know you’ll get to the point,- Then they are more likely to give you some time. BUT THE PRINCIPLES REMAIN – USE SLIDES AS THEY ARE MEANT TO BE USED – AND PAY ATTENTION TO YOUR VISUALS – USE PREATTENTIVE PROCESSING WHERE YOU CANDIAL DOWN the emotional impact of the slides maybe, but do not revert to slidumentsHave the full information available ON PAPER
- Here, the GRAPHIC CARRIES ALL OF THE VISUAL WORKLOAD. But it might be a way to “EASE” your audience into more compelling approaches. NOTE the use of preattentive attributes
- If you really feel you have to use titles and bullets – be MINIMALIST – THESE ARE THE CHARACTERISTICS OF WISE USE OF TEXT SLIDES
- OK – BACK TO THE FUN STUFFHere’s a familiar guy. But here is something that may be an unfamiliar idea. And yet, it is one of the most powerful ideas that we humans have ever had in communicating with one another. It’s called – THE HERO’S JOURNEY
- MAYBE DON’T USE FOR THIS PRESENTATIONSyd Field is considered the godfather of the Hollywood story template. Based on his research and experience, he redid the ancient idea of the “three act play”, and came up with what he called “Syd Field’s Paradigm”He noticed that the Second, or middle, ACT was generally twice as long as Acts 1 and 3. Act 2 contained most of the story, the back and forth and up and down of the plot. Act 2 begins after a “PLOT POINT”, that takes us out of the early set up and into the action. Think of Luke Skywalker finding his Aunt and Uncle murdered by the Storm Troopers, or Dorothy being sucked up in the tornado. Act 2 ends, similarly, after another Plot Point, where “the storm breaks” – the Death Star is destroyed, or Dorothy et al. confront the Wizard and win his gifts. You can see this in just about every movie and play you’ve seen,,,, and it works for presentations. BUT let’s look at something a bit more refined in detail, to relate to presentations.
- Here is the HERO’S JOURNEY,,,Typically, the movie begins with a likeable hero, as yet unaware of an impending problem. The Hero is in his ORDINARY WORLD –This progresses through stages, clock wise, through PLOT POINTS and Ups and Downs, until the final resolution, as in Syd Field’s Paradigm. The part of this journey that corresponds to what you do in a really good presentation is from TWELVE O’CLOCK, in the ORDINARY WORLD, THROUGH WHERE THE HERO ANSWERS THE CALL TO ADVENTURE (PLOT POINT) to the point where the HERO (THE AUDIENCE), decides ON THEIR OWN (PLOT POINT) to ANSWER THE CALL TO ACTION AND CROSS THE THESHOLD (TO DO WHAT YOU ARE PERSUADING THEM TO DO). (GO THROUGH THE STEPS, REFERRING TO THE AUDIENCE’S JOURNEY)(JUST USE THIS AS A WAY TO TALK ABOUT – - YOU ARE PRESENTING YOUR BIG IDEA – YOU NEED TO HAVE ONE,,,,, AND YOU START TO INTRODUCE IT IN THE CALL TO ADVENTURETAKING THE AUDIENCE ON A JOURNEYOFFERING ADVICE, TOOLS AND OVERCOMING RESiSTANCE ONCE THEY HAVE ACCEPTED THE CALL TO ADVENTUREGET THEM TO SEE THE REWARD POSSIBLE FROM ACCEPTING THE BIG IDEAGET THEM TO WANT TO ACCEPT THE BIG IDEA, ANSWER THE CALL TO ACTION AND CROSS THE THRESHOLD – ON THE WAY TO THEIR REWARD
- MAYBE JUST REFER TO THIS QUICKLY AS “S0METHING ELSE YOU’D FIND USEFUL IF YOU LOOK INTO THIS MORE (OR USE IT IF THERE IS TIME)Drawing from the HERO’S JOURNEY, then, is a concept invented by NANCY DUARTE – this describes the “contours” of communication – specifically, persuasive and engaging communication.Duarte analysed many influential speeches and presentations and came up with common elements and structure. At the start,,, there is a description of WHAT IS ( the likeable hero is unaware),,, then going to the revelation of a GAP,, where WHAT COULD BE is seen distinctly. That’s the first TURNING POINT, the CALL TO ADVENTURE. Contrast between what is and what could be then continues, engaging the audience and creating tension and resolution, until a final TURNING POINT is reached where the REWARD available to the audience is clear and compelling,.That’s where the presentation ENDS. The HERO (AUDIENCE) then must decide themselves to ACCEPT THE CALL TO ACTION and CROSS THE THRESHOLD. You are not going to deliver a speech like Kennedy announcing the push to put a man on the moon for your management, when introducing your evaluation findings and recommendations. You will have much less rhetorical and emotional energy. But if you want to be persuasive, the structure must be there.
- Describe the creative process
- http://www.examiner.com/video/reagan-chalenger-disaster-speech
- WE HAVE COVERED A BUNCH OF CONCEPTS AND PRINCIPLES. (GO THROUGH THEM)THERE IS A LOT MORE, and there are some processes and techniques to put this together and make it work. KEY REFERENCES ARESTEPHEN FEWNANCY DUARTEGARR REYNOLDSI’D RECOMMEND LOOKING INTO THEIR WORK
- HERE’S THE THINGWE TALKED ABOUT THE HIGH STANDARDS OF SCIENCE AND RIGOUR IN EVALUATION – FOLLOWING PRINCIPLES OF SCIENCE AND MATHEMATICSAND NOW WE SEE THAT MAYBE SOMETHING HAS BEEN LEFT ON THE TABLE –
- AND THAT IS THAT THERE ARE PRINCIPLES WITH WHICH WE ARE LESS FAMILIAR, THAT CAN BE KEYS TO BRINGING THE PAINSTAKING WORK OF DOING AN EVALUATION TO A TRULY WORTHWHILE POINT – WHERE THE KEY MESSAGES ARE RECEIVEDTHE KEY MESSAGE IN THIS PRESENTATION IS THAT YOU CAN BEGIN TO USE THESE PRINCIPLES TO HAVE THE IMPACT THAT YOUR WORK DESERVES