I argue we have no performance measures for the website. Social media gurus tell me the costs surrounding Edmontonstories are astronomically high and we can’t even gauge if it alone has drawn a single person to come live in this city.

Now, two years later, where are we? Not much further ahead. Here’s the sad reality:

In its first four months, EdmontonStories.ca attracted 113,979 total visits. Five months later, that number had grown to 203,685. And in the two years since, it has attracted just 358,691 more visits, bringing the total to 558,376. Most of the growth took place in the first year! Since a picture is worth a thousand words, here’s a graph to show you what the growth curve looks like (linear and logarithmic):

And remember that those numbers are total visits. There’s no word on how many are uniques. The number of people visiting from outside Edmonton is even less, especially when you consider that when an Edmontonian’s story goes up they likely share it with friends and family in the city.

The number of stories on the site likewise has grown very slowly. The total now sits at 339 compared to 272 in February 2010.

The same case study that was held up in defense of the site two years ago is the one Administration is using now (the EPS one). The report mentions just six organizations that have joined the Recruitment Campaign Partnership. Six! Out of all the organizations in Edmonton!

And yes, the budget is a concern. Incredibly, the report does not make it clear how much has been spent on the project. It does state that $1.5 million was allocated in the first year and that a consultant’s estimate of the “right” investment amount was about $5 million. Councillor Diotte says that with this year’s $600,000 budget factored in, a total of $3.5 million will have been spent on the site since it launched.

$180,000 is being spent to advertise the website to extend its reach, yet we know that the growth rate has declined significantly over time.

$144,000 is being spent on the recruitment program, which has attracted just seven partner organizations in the last two years.

$126,000 is being spent on “managing, maintaining, monitoring and engaging target audiences of various social media platforms.” You know, the stuff you and I do every day for free.

$54,000 is being spent on “research, planning & development.” I’m not exactly sure what this would refer to in the third year of a program like this.

$54,000 is being spent on “website development & maintenance.” I pay $90 per month total to host this site and at least half a dozen others on Amazon EC2. And I can confirm that it more than handles the kind of traffic EdmontonStories.ca has.

$30,000 is being spent to extend the brand into trade shows and other events.

$12,000 is being spent to help people write new stories, yet just 67 new stories have been posted in the last two years.

Clearly the cost is a concern. But perhaps the biggest problem is that the site’s champion is no longer driving the site forward. I don’t think it is a coincidence that after Mary Pat left the City the site received less attention. Reading the report from Administration, it certainly feels like there’s a gap from 2010 until now. It’s hard to look after someone else’s baby.

I recognize that you don’t get results over night and that developing a successful program can often take time. But three years should be enough time to decide whether or not to pull the plug. That’s an eternity in the online world! Incredibly, Administration thinks we should do the opposite by reaching out to more organizations, recruiting student partners, and enhancing the site with things like Google Maps.

I think there’s value in what has been created at EdmontonStories.ca and I believe there are ways to continue to leverage that (perhaps via EEDC, which always did seem like a more suitable home for it), but I don’t think the City should be investing any more into the project.

I came back to read other comments, and realized that I forgot that your name ends with “K”. I get a little eager to hit some times.

http://blog.mastermaq.ca Mack D. Male

No problem

http://twitter.com/shakois shakeyshakois

so what growth would you expect for a site like this? clearly not a straight line from the start, yet that seems to be your expectation. and in place of this, what would you suggest the city do to market itself for $600,000/year or less? do you realize the budget of EEDC over the same period has been $24M? do you think they’ve been marketing the city well?

the argument about doing stuff for free – also incredibly weak. do you really think that would deliver the performance and professionalism you would expect from the city?

always easier to tear down than build up. a little more thought needs to go into this analysis.

http://blog.mastermaq.ca Mack D. Male

Agreed, always easier to tear down. I don’t think the growth needs to look like a hockey stick, but for $150K per year or more on advertising, I’d expect to see something more substantial.

On the point about the professionalism….have you read the EdmontonStories account? Have you read mine?

kcantor

sorry shakeyshaois… mac’s dead on with this one. the entire reason for creating Edmonton Stories was to promote the city of
edmonton. if it needs more promotion -anywhere – in order for it to be
successfull three years later, doesn’t that mean it is a complete demonstrated failure in accomplishing what it
was meant to?

i think this was set up to fail long before it was even executed and looking
at the execution – or changing it after the fact – won’t change that other than
throwing good money after bad. fix those assumptions and do it right as part of eedc or even the city’s home site (and fix it while they’re at) if it’s
going to be continued in any form or close it down – anything else is just
changing the colour of the lipstick.

http://www.ChrisLaBossiere.com Chris LaBossiere

This was a great idea, horribly executed and mired in civic bureaucracy. The idea of organic and authentic stories of Edmontonians, compelling others to visit or move to our city is a worthwhile one. But considering the budget and the forced nature of the end product, I am confident we are not getting our value here.

Sometimes we need to admit our mistakes and try to make the best of a bad situation. In this case, to me that means letting it stay, but transitioning it to a more open-source, user managed solution. I get the fear of content control, but like Wikipedia, I think we can have faith in the community managing it to best represent the City.

http://twitter.com/Drewforward Drew Forward

Hammer – meet nail. I think that $180K could have been better spent towards generating content. That would have given the site a better chance I think.

Disappointed Edmontonian

sorry, have to disagree. I’ve never even heard of this site Edmontonstories until the controversy started this past week. I am one of the hits on the site as I had to check it out myself. It is a total waste of money, in my opinion. When the economy is good people will move here. When times are tough people won’t. Word of mouth and this city sell itself. There is no need for the city to waste any more of taxpayers’ hard earned dollars on that site. We’re being taxed to death. Time to move away!

Graham Simpson

Viral sits in a bottle no genie can summon (that shudda been in sh#t Edmontonians say). It’s hard to force cool. Seems that it’s even harder to cut losses. But if they want to keep it running, I’d happily do it for .01% of what they’re paying now (including hosting costs). Next up? An arena? Cue: face palm.

Richard

Be careful for what you wush for Graham. I’ve ran Connect2Edmonton for years now out of my (and a couple others) pockets…you won’t be “happy”…it is a lot of work for no gainand a lot of abuse…especially if you dare criticize the group think…

I’m seriously thinking on pulling the plug there too…

http://blog.mastermaq.ca Mack D. Male

The difference is that C2E actually has a community, whereas Edmonton Stories is really just a collection of static stories. I think that makes running C2E both more difficult and interesting!

Sorry to hear that you might be pulling the plug there, would love to chat with you about that further if you’re game Richard!

Richard

Always game…

Feel free to reach out anytime…easiest is to PM me on C2E or even FB

http://blog.mastermaq.ca Mack D. Male

Thanks, sent you a message on Facebook.

http://ericwarnke.com Eric Warnke

$3.5m buys a lot of Fiverr work…

But really, the reality is that the City loves its RFP process, and only big expensive companies can afford to play the game in that process, hence $50,000 websites and $126,000 Twitter accounts.

This is right up there with the giant metallic pile of goat turds and the clay feet that are being erected around our fine town.

Marina

why is there not more co-ordination between all the Provincial and City tourism agencies and EEDC. Good that there have been recent mergers to stop the overlap but they really should sit down together and promote each other’s initiatives. The Edmonton Stories idea was great, end result often corny and often so worthy…these stories should have been funny, raw, people can make their own stories these days with an iphone, should be a place for that too on this site. But doing everything for free or on fiverrr is a downward spiral to a lot of people working hard for a pittance.

http://adrielmichaud.com Adriel

358,691 over two years comes to just under 15,000 visits per month, which could be success IF the traffic was right. If you had 15,000 potential visitors/new Edmontonians visiting the site each month, that’d be awesome. If it’s a ton of visits from Google’s Display Network, or random cheap banner ads, then who cares? If it’s a ton of Edmontonians, there’s some sort of value there, but not really what the site was intended for.
The problem is that visits means nothing in the context of the site’s goals.

http://twitter.com/journalistjeff Jeff Samsonow

Is it weird that I get a sense of glee when I see you’ve got a new stats-based story?
Probably.

Excellent analysis, and fair, because you certainly spot the City it’s launch and build-up phases. I really haven’t heard much of EdmontonStories since it’s first year, and your note about it’s champion leaving makes that sound all the more reasonable.

Whether or not EdmontonStories continues in its current form I can see value in paying an editor to monitor local blogs and online postings, curating that content, and pushing that out to cities beyond Alberta. That’s where you’ll find all the stories you need to promote Edmonton. And, if someone passionate about such stories is running the site (as you and so many other regular content producers prove just about daily) there will probably be just as much being produced now, for way more money.

http://blog.mastermaq.ca Mack D. Male

There’s definitely lots of content being created on a regular basis, but it is hard work to gather and package it together. Obviously some of us are working on tools to make that easier, but it’s tough. We need better filters!

http://www.darrenbarefoot.com/ Darren

Fascinating analysis–nice work.

That is a really dreadful outcome for $3.5 million. Now visitors isn’t really the best metric to use to measure outcomes, but they basically turned $3.5 million into 558K visitors. That’s $6.27 a visitor, or at least three times what a well-run site ought to be paying for high-performing, good quality visitors.

I’m familiar with a bunch of projects at this scale, and I’d say they overpaid by roughly 100%.

As you indicate, I’d be more interested in some actual success metrics. Unfortunately, the measurable calls-to-action on the site seem to be few and far between.

http://blog.mastermaq.ca Mack D. Male

Thanks for the comment Darren, that’s some useful context. I wasn’t really sure what an appropriate cost/visitor ratio would be, but my gut was that the ratio for Edmonton Stories was way above what it should be.

http://www.facebook.com/profile.php?id=642295067 Marissa Loewen

I actually thought EdmontonStories was a one-time contest. I had no idea it was still going 3 years later. I don’t know if stories will attract people to live in Edmonton – jobs do. Maybe that money is better spent on programs enabling families to move here once they have a job. It might make the transition easier to influence people to accept the jobs here if they know they have a support system (even if it is small) in place already. If we’re going to be faced with an additional 114,000 (give or take a few) extra jobs in the near future, we could make sure people are able to move here. Access to real estate agencies, temporary lodgings, provide reviews on neighbourhoods like they do businesses on Yelp, etc. Sometimes moving to a new city can be daunting enough to prevent it all together. Especially if it is a new province or country as well.

http://twitter.com/CityofEdmonton City of Edmonton

Hi Mack: I know you’ve been a supporter in the past and also want what’s best for this city, so I appreciate your taking the time to analyze the program.

There are a couple of points to clarify that I think would add to the discussion. One is using overall traffic to measure site effectiveness. We feel the real success of the site comes in the impact it has with a very targeted audience. – people who might be considering coming to Edmonton, and who are considering that through the advice of their social networks. So although overall traffic might not be increasing as fast as it did when the site was new, the fact that we can see people spending time reading particular stories about people like them in Edmonton indicates a powerful outreach. The uptake and distribution of these stories through cultural communities is incredibly quick and broad.

The way Edmontonians use the site will differ from the way Canadians outside Alberta might. This applies to international traffic as well. Although we make considerations towards total visits to help us measure reach, we look closer at engagement stats when performing our optimizations. In fact we make considerations to these stats when applied to each inbound marketing channel.

The other point to be made about traffic is that the initial push for traffic was needed to seed the site with relevant content. Now that’s happened, a more gradual growth makes sense.

The EPS case study is still used in our briefs and materials because it is a proof-of-concept of how the content, social marketing and recruitment partners work together to position Edmonton as a viable option to people considering relocation! We are now moving into Phase 2 which will see us hopefully repeat the efforts and numbers of the EPS case study 10 fold. We are partnering with companies that speak to thousands of people every month and we are supply a tool for them to use to improve their own conversion rates!

Having said all that, let me address the budget breakdown from our perspective:

30% paid online promotion (180k): this is a fixed cost and translates directly into extending the reach. The higher the number, the more targeted, qualified traffic delivered directly to the site, one click at a time. Only qualified clicks are paid for, which is a low risk way to extend the site’s reach outside of Edmonton and into identified markets to ensure every dollar counts. This number is quite low when compared to traditional media buys or campaigns.

24% recruitment program: The partner program was actually only started in the summer of 2011. After working out the details of the program we identified and approached 10 prospective partners. All 10 welcomed the program with open arms and currently 8 of them are online, while the other two are finalizing their participation.

21% social media execution: You are absolutely correct here. Many social media professionals do undertake these activities every day. If you know of any that would like to devote the required number of hours to this project free of charge, let us know!

9% Research, Planning and Development: Since 2009, Facebook has nearly doubled in size, Google has turned their SEO algorithms upside down (sad panda?), smart phone/mobile usage has exploded, twitter is starting to be used by more than just annoying people tweeting about their breakfast and we’ve seen more social media platforms come and go than we care to count. Our point, the online landscape is changing and part of this program is constantly evaluating where our resources are best places to ensure our content delivery vehicle stays relevant. (for interest sake, we’ve seen our organic search engine traffic double since panda!)

9% Website Hosting & Maintenance: Yes hosting costs are a small portion of this budget and so are the tools we use for offsite backups, and to help us combat cyber attacks. The rest of it is dedicated to continually evolving the user interface. This also covers new sections of the site being created, promotions supported and campaign landing page optimization. We have designs in 2012 for a mobile interface and possible deeper Facebook content integration.

2% content support: As mentioned above, far less emphasis on content generation, although we don’t want the content on the site to stagnant, we need SOME new content

As for your comments about Mary Pat – indeed we have her to thank for the concept and the wonderful repository of content we have at the heart of this effort. We continue to try to build this program to achieve her original vision and to evolve it to be effective for the city, and provide value for tax dollars.

The City has received recognition in the form of awards, praise from municipalities for this innovation and there are more and more comparables popping up every week. The content we have now leaves us well positioned to help organizations with their looming (present?)labour recruitment challenges.

Everyone involved with Edmontonstories is as every bit as passionate about improving this city as you. We’re always open to suggestions on improving it, but to stop now would be a real shame and a real loss of money already invested.

I’d welcome the chance to talk with you on this more in person, and will send you an email about getting together.

Thanks for this thorough response, Robert. I can’t speak for Mack, but I’d be very interested in what you’re measuring and what results you’ve seen when you refer to “engagement stats”.

I appreciate that measuring success on a site like this isn’t best expressed in visitors, but as I’m sure you know, people will glom onto the numbers you make available. So, I’d be keen to see some other more meaningful ones.

I’d also suggest that you have somebody audit your online promotions program. If Mack’s numbers are accurate, and you spent at least $180K on online promotions in 2011 (as you plan to do in 2012), you really ought to be getting bigger bang for your buck.

http://www.darrenbarefoot.com/ Darren

I should add that these types of large-scale, multi-year web projects are incredibly hard to get right. I’ve been involved in planning, advising on and resurrecting several, and the best intentions oft go awry. Stakeholders need to make smart decisions about a culture and technology (that is, the web) they often don’t know enough about, and consultants are mostly self-interested.

More importantly, you have to be a clairvoyant futurist, and predict what the web is going to look like in two or three years. And be nimble enough to shift gears as the web evolves.

In short, while I’m sounding critical, the story of EdmontonStories.ca is very familiar, and should be considered a learning opportunity for this and other municipalities.

http://www.facebook.com/profile.php?id=642295067 Marissa Loewen

I just checked out edmontonstories.ca – and you’re right – there is outstanding content on the site. I am surprised it isn’t integrated into edmonton.ca more tightly. I know you have a block on the home page, but it took me a bit to really LOOK for it. I went to the Living Page on edmonton.ca which is prominently in the centre of the homepage. There is no call to action for people moving to Edmonton quickly visible. Cashing in on the traffic that edmonton.ca is getting to not only speak to future Edmontonians, but also current Edmontonians who might be moving elsewhere. When I search in Google – moving to Edmonton doesn’t produce Edmonton.ca (It does Edmonton.com which is a whole other confusing piece of the puzzle for people outside the city) but it does bring a story from Edmontonstories.ca

On these stories with great search keywords, I would add sidebar links to related information about moving, maybe the partner pages, etc.

Maybe it’s time to make the stories relevant to other services the City provides.

I suppose there is so much one can do if we have relevant resources though and if costs are a factor, resources mean people and not just tools that can be found for free. I think an important note that this budget includes salaries of people living and working in Edmonton.

http://twitter.com/journalistjeff Jeff Samsonow

I’d also be interested to know if there are stories from anyone who came here because of EdmontonStories. Seems those should be the ones trotted out to councillors with budget questions looming.

http://blog.mastermaq.ca Mack D. Male

Thanks Robert for the thorough comment. As you probably know I’m always open to a face-to-face chat, and I’m definitely interested in following up on the stories I cover as I learn more. I look forward to chatting with you further!

I do think that Darren’s comments especially should be considered with extra attention by your team. He knows what he is talking about, and is truly a leader when it comes to campaigns like this in Canada.

Edmo2012

I nearly stopped reading when I heard that councillor Diotte is taking advice from “Social Media Gurus”. I am sorry but anybody labeling themselves with that moniker is likely a no talent hack after your money, and anybody listening to them is a sucker. On side not this project was a waste of money.

http://blog.mastermaq.ca Mack D. Male

I agree with you that “social media guru” is a completely meaningless label and any such self-described individuals should be treated with caution!

Graham Simpson

“Social media gurus”are the ones who promise they can pull the genie out of the bottle. They’re only ‘experts’ because they have better reasons for being wrong.

A Canadian Foodie

I am shocked that that kind of money has gone into that site. I find value in the site – and definitely value in the idea – and I hope the information is archived somewhere – maybe with the historical society – but, good grief – give me a break on this kind of money spent on these kinds of results. Were there no bench marks put in place to shut it down when…? or if….?