A/B testing email campaigns – an example

September’s Net Tuesday brought up a lot of questions about best practices for email campaigning.

Long or short subject lines? One ask or newsletter style? Weekly or monthly? Text or HTML layouts?

Now I may not be a grizzled old veteran yet, but I’ve already seen that “best practices” for one organization may not necessarily be the optimal strategy for everyone else. So, you have two choices when developing your email campaigns:

  1. you can guess
  2. you can test

I suggest test! It’s fast and easy with most email service providers.

Example using Mailchimp (but almost any major service will do)

NOTE: this was not a statistically valid experiment because of the small size of the test group. I did this as a test of Mailchimp’s functionality and to show how easy it is to setup an A/B test.

I setup an A/B test on the from field of a recent invitation to the Vancouver Net Tuesday group (instructions). The test sent the two variants of the email to a random 30% of the total list, and then after 24 hours automatically sent the winning variant of the email (based on open and clickthru rates) to the rest of the mailing list.

  • Group A: NetSquared Camp
  • Group B: Eli from Net Tuesday

And here’s the stats on the test:

As the French say, “et voilà”.

Net Tuesday September notes

Thanks to everyone who came out to the Net Tuesday September event focusing on email marketing and fundraising. Your insightful questions were what made it all possible. Well, the awesome audience AND the panelists (Duncan Owen, Shannon Daud, and Ben Johnson.) Thanks all for making me look good!

There won’t be a Net Tuesday in October, but we’ll be back in November. In the meantime, you can relive the summer’s NetSquared Camp unconference with Russell Bennett’s great video.

What I learned this month:

  • Panels are a great format for Net Tuesday events. They have lots of energy and help bring the audience into the discussion because they support Q&As really well.
  • I need to bring in a mic and speakers for all future events. Struggling to hear the presenters sucks.
  • Name tags aren’t solving the “no mingling” problem. I need to bring in food AND assign people to start up discussions before the events.

Email campaigning service providers:

The panel didn’t have time to go deeply into their favorite tools for setting up and distributing email campaigns, but here’s the URLs for the tools mentioned.

For more see Seattle nonprofit consultants Groundwire’s great overview of email service providers with a focus on integration with Salesforce.com.

A-B testing in an email campaign

September’s Net Tuesday brought up a lot of questions about best practices for email campaigning.

Long or short subject lines? One ask or newsletter style? Weekly or monthly? Text or HTML layouts?

Now I may not be a grizzled old veteran yet, but I’ve already seen that “best practices” for one organization may not necessarily be the optimal strategy for everyone else. So, you have two choices when developing your email campaigns:

  1. you can guess
  2. you can test

I suggest test! It’s fast and easy with most email service providers.

Example using Mailchimp (but almost any major service will do)

NOTE: this was not a statistically valid experiment because of the small size of the test group. I did this as a test of Mailchimp’s functionality and to show how easy it is to setup an A/B test.

I setup an A/B test on the from field of a recent invitation to the Vancouver Net Tuesday group (instructions). The test sent the two variants of the email to a random 30% of the total list, and then after 24 hours automatically sent the winning variant of the email (based on open and clickthru rates) to the rest of the mailing list.

  • Group A: NetSquared Camp
  • Group B: Eli from Net Tuesday

And here’s the stats on the test. In this case “Group B” won with the more personal “Eli from Net Tuesday” from line.

As the French say, “et voilà!”

Net Tuesday September notes: email service providers for nonprofits

Thanks to everyone who came out to the Net Tuesday September event focusing on email marketing and fundraising. Your insightful questions were what made it all possible. Well, the awesome audience AND the panelists (Duncan Owen, Shannon Daud, and Ben Johnson.) Thanks all for making me look good!

There won’t be a Net Tuesday in October, but we’ll be back in November. In the meantime, you can relive the summer’s NetSquared Camp unconference with Russell Bennett’s great video.

What I learned this month:

  • Panels are a great format for Net Tuesday events. They have lots of energy and help bring the audience into the discussion because they support Q&As really well.
  • I need to bring in a mic and speakers for all future events. Struggling to hear the presenters sucks.
  • Name tags aren’t solving the “no mingling” problem. I need to bring in food AND assign people to start up discussions before the events.

Email campaigning service providers:

The panel didn’t have time to go deeply into their favorite tools for setting up and distributing email campaigns, but here’s the URLs for the tools mentioned.

For more see Seattle nonprofit consultants Groundwire’s great overview of email service providers with a focus on integration with Salesforce.com.