Let's debate! Impact Metrics: How will we know we are successful?
Hosted by Jeff Skoll, Sally Osberg and Keely Stevenson (June 2003 - Closed)
Keely Stevenson - 01:54am Jun 24, 2003 PSTRoyal Bafokeng Economic Board
As we venture into building this community, we are asking ourselves a very important question:
How will we know we are successful?
Measuring success in this community (or in the sector as a whole) is an interesting discussion and certainly one worth some debate. First, I thought it would be worth it to remind everyone of Social Edge's vision, mission and goals.
Social Edge Vision: To become a connected community fueling advancement and empowerment of the social sector. Social Edge Mission: Social Edge connects the community to provoke dialogue that inspires systemic change in the social sector. Social Edge Goals: 1) Connect social entrepreneurs, their partners and allies to discuss cutting edge issues shaping the field. 2) Foster frank dialogue, mutual respect and a sense of community among all in the social sector. 3) Promote learning from the best, promising and disastrous practices.
Some believe that finding metrics to represent these is a challenge, but I believe we have seen some excellent examples already during this beta phase. We are looking at qualitative and quantitative measurements for success.
What are some of the metrics & criteria you would use to determine if Social Edge is successful in 1, 5 and 15 years? What should we use as indicators of a healthy community with a powerful impact on the world?
What examples of furthering our mission have you seen already (ideas disseminated, relationships built, action taken)?
How will we know we are successful?
Measuring success in this community (or in the sector as a whole) is an interesting discussion and certainly one worth some debate. First, I thought it would be worth it to remind everyone of Social Edge's vision, mission and goals.
Social Edge Vision: To become a connected community fueling advancement and empowerment of the social sector. Social Edge Mission: Social Edge connects the community to provoke dialogue that inspires systemic change in the social sector. Social Edge Goals: 1) Connect social entrepreneurs, their partners and allies to discuss cutting edge issues shaping the field. 2) Foster frank dialogue, mutual respect and a sense of community among all in the social sector. 3) Promote learning from the best, promising and disastrous practices.
Some believe that finding metrics to represent these is a challenge, but I believe we have seen some excellent examples already during this beta phase. We are looking at qualitative and quantitative measurements for success.
What are some of the metrics & criteria you would use to determine if Social Edge is successful in 1, 5 and 15 years? What should we use as indicators of a healthy community with a powerful impact on the world?
What examples of furthering our mission have you seen already (ideas disseminated, relationships built, action taken)?
Anne Perlman - Jun 24, 2003 10:26 am (# Total: 17) Relationships built
Keely, an example of the "relationships built" furthering of the mission that you ask about struck me in Simon Healy's message as part of this event. I've pasted that part of his message below as a color-rich sample of what you're asking. Thanks to Simon for this!
Anne ------------
In the case of Social Edge here's something that occurred last week.................
I posted a small hello in 'Are you a social entrepreneur discussion.' Michael Chertok who runs an ICT centre (www.digitaldividedata.com) in Cambodia responded and emailed me to see if we could do some work together. I responded that we could and were having a phone call later this week to discuss things. Whilst this was happening I noticed some people in Michael's organisation were also attending the Global Knowledge Partnership's INFOSOC conference in Malaysia this week as a sponsored best practice ICT and we also have one of our project leaders (Dale) there as a sponsored presenter. I told to Dale to keep an eye out for Gordon (Michael's team member) because Michael had mentioned this to me. However Dale had already met Gordon, they have become mates and started discussing working together. This is whilst Michael and I are working out how we can work together to create more work at his centre. = Beautiful!
Social Edge was the catalyst for making what I hope to be a long term relationship happen between OrphanIT and DigitalDivideData >> Outcome = Job creation, poverty alleviation for many youths at Michael's centre in Cambodia.
So Social Edge has probably contributed to solving poverty in the long term for a small group of people in Cambodia
Sorry that doesnt help with your transaction issue but Social Edge is certainly providing social transactions and a wealth of other benefits that will keep people coming back if more and more relationships like this can be successfully created around the world
Bravo to all of you and long life to Social Edge. --------
K.L.SRIVASTAVA - Jun 24, 2003 9:35 pm (# Total: 17) Researcher and Consultant,Hyderabad,INDIA
Impact metrices
I envision that Social Edge will evolve as a global network of interacting regional hubs who will have regionally-relevant as well as global agendas. The issues like education, health, and environment will form the dominant streams of opportunities for social entrepreneurship, and hence, the discussions are likely to be structured along those themes. I can think of following measures of success: (a) satisfaction level of participants about what they have gained in terms of ideas and contacts for their personal missions and persuits. (b) satisfaction level of participants about opportunities they got for contributing their perspective and knowledge. (c)Scoring on relevance of discussions in relation to contemporary issues. (d) Feedback about Social enterprises formed/developed due to participation in this forum.
Thanks
Jeb Brugmann - Jun 25, 2003 8:36 am (# Total: 17) Globalegacy
President, Globalegacy
Greetings All! Regarding metrics for SocialEdge, perhaps we could add a metric on the service's methodological contributions to the state of practice in the social enterprise sector. One area where the sector needs to continue to develop is, in fact, with regard to the metrics used to evaluate the sector's impacts.
Globalegacy wants to avoid the erroneous assumption that building a successful social enterprise, with a solid triple bottom line, will necessarily have any substantial positive impact on the fundamental (generally negative) economic relationship between a low-wealth community and its broader, more affluent metropolitan economy. That relationship is generally characterized by leakage of money/wealth/human, social and natural capital from the low-wealth community to the affluent market. We are presently working to develop a metrics framework that places greater emphasis on tracking changes in this fundamental economic relationship, and its patterned negative flows. The question we face is, how can social enterprise be used to turn those flows in favor of the low-wealth community?
With the right metrics we can design social enterprises that are able, through their business processes, to transform patterned, negative relationships that are reinforced by various institutional arrangements. We know that this will require:
--a user-friendly methodology for modeling the economic fundamentals of a low-wealth community and its money/capital flow relationship with the metropolitan/regional market (currently being developed in partnership with New Economics Foundation and Monitor consulting group);
--use of this economic analysis in designing individual social enterprises AND THEIR INDIVIDUAL FIRM-LEVEL METRICS, so that the enterprise can change the institutional foundations of patterned negative relationships;
--employing a density strategy for enterprise development, so that the partner is a low-wealth community, working to transform its economy, not just the entrepreneur/individual enterprise. (Towards this end, we make a long-term commitment to establish a local enterprise cluster of, say, 10-20 firms.
Overall, I think it would be a great contribution if SocialEdge could facilitate sharing of practice in this area so that we can shift from claiming success on the basis of individual (and often geographically isolated) enterprises to evaluating success on the basis of total social enterprise cluster or sector impacts on the fundamental economic patterns confronting a low wealth community.
I very much look forward to this exchange!
Jeb
Steve Rudolph - Jun 25, 2003 8:52 am (# Total: 17) Director, Jiva
Measuring Success
Hi Keely,
I am most of all inspired to see you thinking for the future--15 years. That says a lot about your commitment and vision. Most folks can't see past lunch ;)
I like KL Srivastava's points very much. I'd like to suggest a few more:
- life span of users. Do they come for two weeks and disappear, or do they stay involved for 15 years?
- its internal priority at Skoll. How central a role does it play in your scheme of things as time passes?
- recall to use Social Edge. When something critical happens, do you reach for "The Edge"? (when I search, I "Google", when I buy, I "eBay", When I hit a social entrepreneurial road block...).
- Specifically, how much do people reach their goals and impact their communities more effectively as a result of interactions on SocialEdge?
- How many "Trackback" links are made to your site/postings. In the Weblog world, there is a "trackback" feature that enables people to link postings from your site to theirs, and your site can display how many links have been made, and where to. It is something that will likely take time to become popular, but in a few years, it might be common.
- If people check SocialEdge as much as (or more than) their email
- If people ever have a dream about SocialEdge (then you KNOW you're there)
sally osberg - Jun 25, 2003 10:10 am (# Total: 17) CEO of Skoll Foundation
re: Measuring Success
Hi Steve,
Does it count if the Skoll team is dreaming Social Edge? If so, we're there!
I think it important to respond to your question regarding how central Social Edge is and will continue to be to the Foundation: you should know that as recently as a week ago, Jeff identified Social Edge as the Skoll initiative with the greatest potential to make the greatest impact.
It's a keeper, and we're here for the long pull. But it's people like you who will really nourish the site as you use it, draw inspiration from it, offer ideas and experiences--in other words, continue doing what you're already doing.
Many thanks.
Sally
Steve Rudolph - Jun 25, 2003 11:45 am (# Total: 17) Director, Jiva
re: Measuring Success
Why yes it sure does count if the Skoll folks are dreaming about it! When something grabs you so deeply, and you are so commited to it, it becomes infectious...and others will start dreaming about it along with you.
It seems like you guys are exhibiting Class A signs of "social intrapreneurship" over there :)
I'm wondering if that is a rare thing in foundations. (Social)entrpreneurs thrive on the excitement of innovation, uncertainty, chaos, and achieving their mission. Do foundations typically cultivate a spirit of entrepreneurship/intrapreneurship among their staff members? It seems that such practices would have the potential to stimulate innovation at the foundation level, and also build bridges between them and social entrepreneurs. Maybe social entrepreneurs could even provide a few tips to the funders!
And perhaps things wouldn't have to be viewed as "top down" all the time. We could see ourselves all as equals (the donors, the foundations, the NPOs, the communities we work with), each having a different, but critical role to play in the process of social "upliftment".
Blind Babies Foundation
Depends on Area
It is interesting to me that all of the respondents have hit on the various different types of measurements: 1) Short term quantitative 2) Short term qualitative 3) Long term goal measuring
Your measurements should relate to your goals. They should measure how far you move in relation to each goal. Steve Rudolph makes an excellent suggestion of measuring loyalty quantitatively over time. K.L.Srivatava has some valid suggestions for satisfaction surveys which may help refine and answer questions that could arise from Steve's metrics (if for example people were not active in the community you would want to know why and question their satisfaction with the site). Ann Perlman points out the tremendous value of the qualitative perspective that gives value to this as a means to the end of the ultimate goal of improving our society. Jeb Brugmann puts it in global perspective - any metrics must be analyzed by target segments. If this community is active only in developed countries it fails.
For thsi excercise I would suggest looking to short term goals and then pose the question of what measurements will tell us that we are moving in the right direction. The goals you listed are below with subpoints of what your interim steps and measures might be: 1) Connect social entrepreneurs, their partners and allies to discuss cutting edge issues shaping the field. a) Define the target market and focus recruitment (get peopel to show up and sign up). Set a goal of so many in year one b) Track issues discussed and categorize them - ask a committee to give a qualitative assessment on the "cutting edge" quality of each discussion. 2) Foster frank dialogue, mutual respect and a sense of community among all in the social sector. a) Metrics for community are loyalty, engagement/participation, individual self-assessments (I would suggest standard online surveys of a random sample early and often) 3) Promote learning from the best, promising and disastrous practices. a) Cultivate archives of best and worst practices - measure postings and access to postings. b) Survey users on helpfulness.
Remember, metrics tell you things about what and how you are doing. They are only helpful if you are prepared to change in reaction to what they tell you. If not enough people are signing up or are engaged then we must be prepared to do things differently.
By the way, this is my first posting but I am so enamoured of this idea I feel very at home in this community.
Yours, Geoff Benjamin
Jeff Skoll - Jun 25, 2003 2:25 pm (# Total: 17) Founder of Skoll Foundation
re: President, Globalegacy
Hi Jeb -
Your post is very interesting regarding social enterprise and its relationship to poor/rich communities. I have a couple of questions:
1) What is a triple bottom line social enterprise? Does that mean a company that is successful on social, economic and environmental levels?
2) It sounds like you are saying that typical firms will tend to make the affluent parts of a community more affluent and poorer parts of a community more poor, is that correct? (I'm trying to understand your thought that where you say that the relationship is "generally negative").
Thanks.
Peter Tavernise
Archiving and knowledge management
Some of those posting have mentioned that they would like to see best/worst practices or solutions archived on Social Edge.
The folks from http://www.knowledgeinthepublicinterest.com have mentioned that their nascent web site may be able to back-end code the data such that it can be more easily searched. Perhaps such technology could be employed by SocialEdge? I haven't the technical details, but I'm sure you could contact K-Public to learn more.
This leads us down the path of discussing knowledge management for the NPO sector--something that needs its own thread.
Best,
Peter
sally osberg - Jun 25, 2003 4:57 pm (# Total: 17) CEO of Skoll Foundation
re: Archiving and knowledge management
Hi Peter,
Good point: you can start a new thread by using the add a discussion element. Of course, you can start it here in "Events" or use "Soapbox" or one of the other sections.
Sally
london calling - Jun 25, 2003 5:05 pm (# Total: 17) re: Archiving and knowledge management
Hi Peter,
There's actually a discusion on best/worst practices already. You can view the forum here:
http://www2.socialedge.org/?14@120.KontanEAaHO.28542@.ee80956
Currently Social Edge also has a search section, but we always welcome advice on making it easier to use. Thanks for the link, I'll see what they have...
Anne Perlman - Jun 26, 2003 2:31 pm (# Total: 17) re: Depends on Area
Geoff,
Thank you for such thorough, valuable suggestions for measurement that relate to the Social Edge goals. You'll see many of your suggestions implemented over time. Please keep on posting, and know that you are at home in the Social Edge Community!
Anne
Keely Stevenson - Jun 26, 2003 2:49 pm (# Total: 17) Royal Bafokeng Economic Board
re: Archiving and knowledge management
Peter, thanks for the site- Knowledge/Content management is an important part of any site, and we are working hard to make this a priority on Social Edge. This is a good opportunity to introduce our search tool:
http://www2.socialedge.org/?searchResults@859.VVcLaaSLRN7.190@.ee6b280
We have put a lot of time into building this search tool and would love anyone's feedback on how to make it more useful!
On a side note, we do have a best, promising and disastrous practices forum here at Social Edge moderated by Karen Nemsick who has done a fantastic job thus far.
Keely Stevenson - Jun 26, 2003 3:32 pm (# Total: 17) Royal Bafokeng Economic Board
re: Depends on Area
Hi Geoff,
Excellent post! So glad you feel right at home because you should. This is your community & you obviously understand the mission so well! Great advice on the metrics- you will be happy to know that we are working hard already to implement many of K.L., Jeb, Steve and your valuable suggestions. Keep them coming! My sincere promise to all of you is that WE WILL LISTEN.
I wanted to chime in with Anne's message about the value of qualitative metrics by sharing with everyone an example of successful existing jewels in the Social Edge community.
As many of you know (K.L. especially), our Charter partner Ashoka/Changemakers co-sponsored a Water Forum on Social Edge to aggregate global knowledge and experiences managing water to help communities become more effective stewards of water resources. Both Quality and Quantity were extremely impressive in the dialogue, but what I am most impressed and proud of is the impact-- these global relationships were built and strengthened (the gift!), important knowledge was disseminated (innovative water solutions framework created) and they are in the works of creating a Water Action Guide where these social entrepreneurs can demonstrate some of these community learnings in the field. That is the true power of a learning community! Those are the types of examples that I feel will make Social Edge a success~~~ exceptional community members like each of you who believe in making a positive impact on the world.
Link to Archived Water Forum:
http://www2.socialedge.org/?14@42.7ocQaMOsa0A.33518@.1ad42e84
jimfruchterman - Jun 27, 2003 10:24 am (# Total: 17) Benetech
Metrics
One of the great things about internet-based projects is that some metrics simply fall out: you have to work hard to not look at the logs and see what's happening. I really enjoy looking at our log data every couple of weeks and seeing activity levels, the countries people are coming from and the like. One of the problems with our field is metric mania: we spend a lot of time debating metrics and methodologies to the point where people are almost embarrassed to be straightforward about simple metrics. I think we should always capture the easy data (trends in number of users, etc.) and then make a cost/benefit decision about some of the fancier data we could capture.
Ecotrust Canada
re: President, Globalegacy
Greetings all, it's quite a privilege to be able to get a window on some very good thinking about these vital issues. I am particularly intrigued by the notion of a "density strategy for enterprise development" as described by Jeb Brugmann.In the resource-dependent communities where I work (in coastal British Columbia), the economy has been narrowly built around large, extractive resource companies (forestry, fishing, mining, etc), and declines in any one sector can have rapid and dramatic negative consequences for communities.
It seems to me that we can draw lessons from nature, whose resilience lies in its diversity. Conservationists attempt to protect biodiversity in order to maintain conditions for ecosystem health and vitality. Surely, an economy is no different: reliance on a single "species" of economic activity imperils the ECO-system. Conversely, economic diversity offers the prospect that - even if one or two enterprises fail - the community can absorb those failures, as long as enough other enterprises (species) succeed and/or adapt. If this sounds simplistic, I apologize - certainly, it seems too complex a notion for many politicians (and industrialists)! If you accept that there is validity in this notion of multi-enterprise development, then it seems to me that Jeb's idea about a density strategy takes on special significance. And then, how do you connect those dense clusters to each other? Especially in isolated rural communities with little access to the services that larger communities take for granted, simple networking of best practices and of innovation becomes a huge hurdle.
As for the link between the rural (producers) and urban (consumers), this is a fundamental idea that deserves much greater attention. Jane Jacobs wrote in The Nature of Economies of building "reliably prosperous communities" and one of the underpinnings that she thinks is essential is to build robust and positive relationships between the cities and rural communities. Instead, we are increasingly seeing the emergence of "city states" that are essentially whole entities unto themselves, most favored nations so to speak, whose wealth almost seems to exemplify that awful line of Gore Vidal's ("it is not enough that you succeed, others must fail").
In Canada, John Ralston Saul has stated that he believes the greatest challenge to this country is not the threat of Quebec separating or even of us being overwhelmed by America to the south. Rather, he thinks the country is being sawn in two - the prosperous communities of the south totally disconnected from (and therefore not invested in) the North. I'm not sure that this message helps shed any light on your discussions, but certainly I believe that unless we invest in a revival of our rural economies through enterprise and social development, we are going to suffer increasing disparites in wealth, assets, and community health and vitality...
Ian Gill
Jeb Brugmann - Jun 27, 2003 11:07 am (# Total: 17) Globalegacy
re: President, Globalegacy
Hi Jeff: Thank you for your reply and questions, and for this forum generally. Let me start with the second question.
Our hypothesis, which is supported by considerable field experience and economic theory, is that most poor communities suffer from patterned leakage of money and capital to affluent markets and institutions. This leakage is typically reinforced by local/national institutional arrangements. Hernando de Soto, for instance, is addressing the effects of legal institutions on this leakage dynamic in detail. At a more general level, and as you know, 'poverty economies'--whether at the national or local scale--suffer from trade relationships that leave them exporting their raw capital/materials and re-importing value-added products. This results in a patterned negative cash flow for the poverty economy, which is itself reinforced by different local institutional arrangements. Of course, every community has faces its own unique challenges as well, including issues of land tenure, land ownership, culture and caste, policy-based barriers to market access etc.
Therefore, social enterprises can be successful businesses and provide a needed local service while at the same time having minimal impact on the overall capital and money flows in the economy. So the individual enterprise may succeed, but the local poverty condition will remain. Recent evaluations of micro-enterprise have exposed this seeming paradox of success in the context of continuing (or worsening) poverty. To overcome this we think that we need to better design social enterprises so that they reverse negative flows and the reinforcing institutional arrangements.
In general, the firms therefore need to be export-oriented and to preferably export those goods or services to affluent markets, creating what I call corrective market access. Doing this will tend to mean that the enterprises may need to be scaled to medium-size in order to handle more complex technology, design, marketing and production/delivery challenges. However, to really change the institutionalized negative patterns on a sustainable basis (in the face of traditional business competition) these firms may need to be supported by local clusters, much as new industries in the affluent economy emerge through clustering. (Towards this end, we are working with the Monitor consulting group.)
So the social venture task then shifts to making a longer-term commitment to specific communities and to developing a cluster of firms in those communities that together can build a more advantageous relationship between its local economy and the globalized marketplace.
Finally, this only works long-term, we think, if local community institutions (a community bank, community development corporation etc.) and/or households hold equity in the firms. Otherwise, the more successful firms, responding to the logic of the market, will relocate operations during the process of growth and consolidation, with possible net negative effects, including relocation of the best human capital. Poor communities that invest substantial social capital in an enterprises success should secure an ongoing revenue stream from that success. Towards this end, we aim to offer workers, community institutions etc. warrants, which can be exercised at low cost, thus also building local investment skills and culture.
Regarding triple bottom-line, I refer to it only generally. I think the concept is most meaningful when it refers to firms that pro-actively seek, as a core business mission, to create added financial, social and environmental value. But the main point is that one firm's triple bottom-line success does not necessarily address the more macro-level dynamics between the slum economy and the metropolitan/national economy.
We're putting all this to the test now in East London, and having honed our business development process there a bit, hope to start working with partner communities and entrepreneurs in South Africa before year-end. In the meantime, we are working with Monitor consulting group and the New Economics Foundation to develop a user-friendly methodology for modeling the money and capital flows between a poor community and the broader market, thus aiding the design of local enterprises that target the reversal of any negative patterns.
Onward and upward! Jeb






