Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

15 September 2010


I've been crafting a list of tools as I consider how I might support individuals who are new to design. Since I'm teaching undergraduates this fall, this inquiry has become even more relevant. Many students are worried that they aren't taking the right path and fear that they might make a mistake in their education/career choices. Haven't we all? In my estimation, some of this stems from the temptation to compare oneself to another and assume that in order to be a good designer, I need to be like ______________.

In light of this, I reflected back on my own journey and asked myself how I got to the place I am today. Looking back, I can't underestimate the value of knowing the often overlooked skills that don't fit into a typical "list of tools." The reason? They are unique to each individual! While I am a proponent of developing and nurturing one's craft, there is also great value in actually knowing and identifying your strengths.

In the words of Socrates: Know thyself.

Some might classify these as "soft skills" but I would suggest they are fundamental to developing your toolkit. And since design operates as a collaborative discipline, understanding strengths can help build strong and effective teams.

I went back to some notes and readings from my own search and came across a few tools that have helped me frame these "soft skills" for myself and others. They may be obvious options but in re-reading them, I was encouraged by the direction my work/life/career has taken.

1. StrengthsFinder
StrengthsFinder proposes that we should reduce the amount of time we spend worrying about what we aren't good at and instead nurture what already comes naturally. Here are my top 5 strengths discovered in this process:

a) Ideation
People who are especially talented in the Ideation theme are fascinated by ideas. They are able to find connections between seemingly disparate phenomena.

b) Connectedness
People who are especially talented in the Connectedness theme have faith in the links between all things. They believe there are few coincidences and that almost every event has a reason.

c) Strategic
People who are especially talented in the Strategic theme create alternative ways to proceed. Faced with any given scenario, they can quickly spot the relevant patterns and issues.

d) Input
People who are especially talented in the Input theme have a craving to know more. Often they like to collect and archive all kinds of information.

e) Intellection
People who are especially talented in the Intellection theme are characterized by their intellectual activity. They are introspective and appreciate intellectual discussions.

2. Myers-Briggs
We've likely all taken this test! Here's a brief breakdown of my INTJ-ness:

With Introverted Intuition dominating their personality, INTJs focus their energy on observing the world, and generating ideas and possibilities. Their mind constantly gathers information and makes associations about it. They are tremendously insightful and usually are very quick to understand new ideas. However, their primary interest is not understanding a concept, but rather applying that concept in a useful way.

In education they are most often found at the college and university level. In the professions, they may be a lawyer, a business analyst, or strategic planner. Some have a strong artistic/creative bent and may become an artist, inventor, or designer. Whatever they do, they do it with intensity.

Final Note: There are many other tools one could draw upon! These are but two that have offered me some language to articulate skills that might be missed in talking about a "toolkit" (and were actually assessed more than 10 years ago). I believe that this type inquiry serves to help a designer acquire the most important tool in these times: a unique voice that will speak to the complexities of our world through the discipline of design.

Photo credit: Martin Whitmore

14 February 2010



The last time you made something, did you ask 78 questions about it? With that many to go through, one can't help but pause and consider the impacts of design.

78 Reasonable Questions to Ask about Any Design
by Stephanie Mills

Ecological
1. What are its effects on the health of the planet and of the person?
2. Does it preserve or destroy biodiversity?
3. Does it preserve or reduce ecosystem integrity?
4. What are its effects on the land?
5. What are its effects on wildlife?
6. How much and what kind of waste does it generate?
7. Does it incorporate the principles of ecological design?
8. Does it break the bond of renewal between humans and nature?
9. Does it preserve or reduce cultural biodiversity?
10. What is the totality of its effects - it's "ecology"?

Social
11. Does it serve community?
12. Does it empower community members?
13. How does it affect our perception of our needs?
14. Is it consistent with the creation of a communal, human economy?
15. What are its effects on relationships?
16. Does it undermine conviviality?
17. Does it undermine traditional forms of community?
18. How does it affect our way of seeing and experiencing the world?
19. Does it foster a diversity of forms of knowledge?
20. Does it build on, or contribute to, the renewal of traditional forms of knowledge?
21. Does it serve to commodify knowledge or relationships?
22. To what extent does it redefine reality?
23. Does it to raise a sense of time and history?
24. What is its potential to become addictive?

Moral

25. What values does its use foster?
26. What is gained by its use?
27. What are its effects beyond its ability to the individual?
28. What is lost in using it?
29. What are its effects on the least person in the society?

Aesthetic
30. Is it ugly?
31. Does cause ugliness?
32. What noise does it make?
33. What pace does it set?
34. How does it affect quality of life (as distinct from standard of living)?

Practical
35. What does it make?
36. Who does it benefit?
37. What is its purpose?
38. Where was produced?
39. Where is it used?
40. Where must go when it's broken or obsolete?
41. How expensive is it?
42. Can it be repaired? By an ordinary person?
43. What is the entirety of its cost-the full cost accounting?

Ethical

44. How complicated is it?
45. What does it allow us to ignore?
46. To what extent does it distance agents from effect?
47. Can we assume personal, or communal, responsibility for its effects?
48. Can its effects be directly apprehended?
49. What ancillary technologies does it require?
50. What behavior might it make possible in the future?
51. What other technologies might it make possible?
52. Does it alter our sense of time and relationships in ways conducive to nihilism?

Vocational

53. What is its impact on craft?
54. Does it reduce, deaden, or enhance human creativity?
55. Is it the least imposing technology available for the task?
56. Does it replace, or does it aid, human hands and human beings?
57. Can it be responsive to organic circumstance?
58. Does it depress or enhance the quality of goods?
59. Does it depress or enhance the meaning of work?

Political

60. What is its mystique?
61. Does it concentrate or equalize power?
62. Does it require, or institute, a knowledge elite?
63. Is it totalitarian?
64. Does it require a bureaucracy for its perpetuation?
65. What legal empowerments does it require?
66. Does it undermine traditional moral authority?
67. Does it require military defense?
68. Does it enhance, or serve, military purposes?
69. How does it affect warfare?
70. Does it foster a mass thinking or behavior?
71. Is it consistent with the creation of global economy?
72. Does it empower transnational corporations?
73. What kind of capital does it require?

Metaphysical

74. What aspect of the inner self does it reflect?
75. Does it express love?
76. Does it express rage?
77. What aspect of our past does it reflect?
78. Does it reflect cynical or linear thinking?

Image source: askascientist.org

02 September 2009

Human-Centered Evaluation

Some folks led a session at SoCap09 that I wish I could have been sitting in on. Twitter allows you to catch a glimpse of the topics but I would have loved to hear the dialogue beyond the 140 characters.

Two people I follow on Twitter were on this panel: Aaron Sklar & Tatyana Mamut (both work at Ideo and are connected to designing for social impact).

Here's the description:
Social investors and social entrepreneurs have been struggling for years to align on the right tools for measuring new-to-the world offerings. These offerings often take years to show results and many initiatives run the risk of stalling or failing due to lack of demonstrated progress. In addition, the evaluation of new solutions is often uncharted territory in terms of how to measure and what to measure. In this session aimed at funders and social entrepreneurs, Ideo, Jd Power and associates and Keystone will share their frameworks and experiences for creating a strategic measurement portfolio based on human centered measurement and evaluation methodologies. In the workshop portion of the session, participants will break into teams and create a human centered evaluation strategy to meet their current measurement challenges. Participants are invited to come with an evaluation challenge of a new-to-the-world initiative that they would like to work on in the workshop. The session is part of a collaboration between Ideo and Good magazine to advance dialogue about evaluation in the social sector.

Twitter began feeding these 5 principles for making change happen:
1. Put people at the center of evaluation
2. Take a systemic view
3. Navigate uncertainty
4. Zoom out to a portfolio view
5. Measure what you care about


Having pursued a focus on human-centered design in my graduate studies, I can relate to the struggle to adequately evaluate. There are so many factors that play into this! Notably, this becomes difficult when you are seeking to access evaluations from those who don't share the same language but who could benefit from your offering. Professor Ranjan at NID has a great diagram that reinforces how to keep people at the center of the evaluation by constantly revisiting the community at all stages of design. His diagram infers being present to these individuals: "You have to get your hands dirty on the ground to be able to really understand your customers' needs." (Lucky Gunasekara)

What Ranjan's diagram can't address is how you should interpret the feedback you receive in order to establish some sort of metric. It likely presumes that a conversation has occurred that will allow you to move onto next stages of evaluation. To me, this presents another reason why meaning and metrics are worth investigating. I continue to return to Jacqueline Novogratz's words in The Blue Sweater where she assesses how social programs have often created a culture of charity that don't empower people to say what they really mean. How can we access what people really think when money (or lack thereof) might affect their decision? Check the Twitter feeds under #socap09.ideo for much more on this. The diversity of discussion is worth perusing.

I'm definitely not a pro in the area of social innovation and/or its measurement. I am keen to grow in it because of the witnessed frustrations of ineffective offerings. I don't want to suggest simplistic ideas on such a far-reaching topic either; my limited experience continues to remind me that the number one principle suggested in this session is the one which will define the remainder on the list. Organizations like the ones included on this panel are asking good questions in order to link us to the broader topics of democracy, transparency and governance (for the sake of those who aren't physically present to lend their voices to their own social innovation but who are the beneficiaries of these discussions on some level).

But that's another blog entry altogether.

For more on this discussion, check out the Innovation in Evaluation forum.