Fixes for bugs/gotchas in Parse’s SDK

Parse did a great job abstracting away persistence, and building on top of Backbone.

However, building an app on top of their code isn’t perfect.

There are some lingering Backbone issues that were outside of Parse’s scope to fix, and the way their code works introduces some other gotachas. These are the ones I found, and their fixes:

  • Anonymous users, namely persisting anything related to a user before the user signs up, requires you to secretly first sign the user up with a fake/random username and password. It’s okay, when they go through the real sign up flow, you can change the username, email and password to real values — associated models will stay related.
  • If you subclass your models (wrapping the native Parse.Object with extra logic), when you fetch those models through their parents, Parse will give you back instances of the native Parse.Object. (Fetching via queries, to which you supply the subclass as the type to query for are okay)
  • Because Parse likes to return native Parse.Objects, and not your domain-specific classes,  .get()’ing related models requires a special getter to make sure what’s returned is an instance of the right sub-class.
  • Parse’s Parse.User model is a special case. Subclassing it requires you to overwrite .prototype. directly.

Code fixes for these issues can be found in this gist:

Identify VIPs in your user base or beta list

As your user base or beta list begins to take off, odds are there’s a few important people in there you could benefit from reaching out to.

But how do you find them, in a list of 1,000 or more, when all you have is email addresses? You use these vip-scanner scripts!

The repo includes two scripts.

  1. rapportive.py, modified from Jordan Wright’s original. This translates each email address into identity information, like name, company and role and links to important social profiles (currently LinkedIn, Twitter and AngelList). 
  2. followers.coffee, which adds the number of followers each user has on Twitter and AngelList.

As a good rule of thumb, users with more than 500 Twitter followers or 100 AngelList followers have noteworthy audiences.

Run these on your email list, then go ahead and say hello to the influencers who already love you!

“What made the graph jump that week?” EpochChart shows you.

Mark important events on top of your data, using EpochChart

What did we do to double traffic back in October?
Why did people start sharing twice as much in early November?  

Questions like these are important to ask if you want to learn from your experiences and get better at guiding your product, and it seemed that one way to give meaningful context to the spikes, flatlines and accelerations in your data would be to compare those shapes of the graph with what real-world events we know to have happened that day/week.

Strangely, there weren’t any tools that supported plotting arbitrary events data on top of a trend line, so I made EpochChart.

EpochChart is a jQuery plugin. It leverages Highcharts, so it’s powerful and feature-rich, but provides a much friendlier API that supports the 80% use cases.

If you use an analytics dashboard to understand what’s happening in your product, do yourself a favor and add a layer of meaning to your data by annotating real-world events in the same space.

EpochChart can be used as simply as:

$(‘#chart’).epochchart(linedata,  events);

Try it yourself. The code’s on GitHub: https://github.com/jfeldstein/jQuery.EpochChart.js

It’s free. Ping me on Twitter if you want to chat about it.

If you’d like to see how it’s made, read the EpochCharts technical write-up.

How and why I made EpochCharts.js

  • This post has all the technical details. If you just want to see what EpochCharts does, and how you can use it to learn from your data, read the announcement post.

Why make EpochChart?

One of the teams I work with wanted to see how their product’s growth numbers compared to the timing of real-world milestones.

It’s easy to graph those numbers on their own, but marking up those charts with any extra data was something no existing libraries did well.

To get the type of visual we wanted, we needed something custom. Thus EpochChart was born.

How was it made?

EpochCharts is a jQuery plugin for graphing line data, overlaid with arbitrary events. Use it to add real-world context to your numeric data, by tagging charts with the timing of real-world events.

Existing libraries we tried before making EpochCharts:

This isn’t something most charting packages can do out of the box. For example:

  • Flot has annotations that can be added to a chart, but the text is written onto the face of the data, which will get crowded with a number of annotations. 
  • Chart.js has a very nice API for rendering data, but no way to mark up the line with named events.
  • D3.js would probably work, if you wanted to build everything from scratch.
  • Highcharts (which EpochCharts is built on top of) doesn’t have out of the box support for annotating a line chart with extra events, so we hacked that on.

Adding Event Markers to Highcharts:

Highcharts does not support a line graph, with arbitrary or intermittent markers, out of the box, but it is very power and flexible.

To get the functionality we wanted, we start with a basic line:


$(this).highcharts({
  series: [...]
})

Removed its markers, using:


$(this).highcharts({
  plotOptions: {
    spline: {
      marker: {enabled: false }
    }
  },
  series: [{
    type: 'spline',
    data: [...]
  }]
});

Then, added markers for each event back on, by overlaying our line data with a custom-built scatter plot:

(That's right, the markers aren't actually on the lines, they're just scatter points placed at the right x,y coordinates)

$(this).highcharts({
  plotOptions: {
    spline: {
      marker: {enabled: false }
    }
  },
  series: [{
    type: 'spline',
    data: [...]
  },
  {
    type: 'scatter',
    data: markerData
  }]
});

Making it look right took a little love:

Putting custom markers in the right places

Since the events we want to tag are independent of the data points making the line, we need to calculate the effective Y value at which to render the marker so that it attaches to the line.

  tallestPoint = function(lines, x) {
    var computedValues, ret;
    computedValues = $.map(lines, function(line) {
      var prevPoint, ret;
      prevPoint = null;
      ret = null;
      $.map(line['data'], function(datum, i) {
        var slope, xdiff;
        if (x >= datum['x']) {
          prevPoint = datum;
          return;
        }
        if (x < datum['x']) {
          if (prevPoint === null) {
            ret = datum['y'];
            return;
          }
          xdiff = x - prevPoint['x'];
          slope = (datum['y'] - prevPoint['y']) / (datum['x'] - prevPoint['x']);
          ret = prevPoint['y'] + slope * (x - prevPoint['x']);
        }
      });
      return ret;
    });
    ret = null;
    $.each(computedValues, function(i, v) {
      if ((v != null) && (!(ret != null) || ret < v)) {
        return ret = v;
      }
    });
    return ret;
  };

And we had to make the image for the marker twice as tall as the marker actually appears, because Highcharts tries to "center" the marker over the X/Y coordinate, and we wanted something that looked like a pin attached to the line.

Taming Highchart's default tooltips

By default, Highcharts uses some smart behaviors that, in most situations, make the tooltips very useful and intuitive. Applied to what we're doing with Highcharts, however, those behaviors tend to get in the way more than help.

We changed tooltips by:

  • Making them fixed positioned in the top left by default, and exposed an option `tooltip: {x:..., y:...}` to set your own custom placement.
  • Customizing the contents, using Highchart's native `Tooltip.formatted` option. The new format puts the date in the header, and looks for either the series name and Y-value of a data point, or the content of the custom marker as the body.
  • Disabling shared tooltips, which stops the whole chart body from acting as a hotspot, so tooltips only show when you're hovering closely over a specific point.

The result is now tooltips that appear when you expect, and display a tidy message in the upper left as needed.

Packing it up:

The plugin is written in coffeescript, which normally wants to add a layer of sandboxing to each file. To break out of it, I used  this simple trick from jashkenas on Stackoverflow.

That's it!

To use EpochChart, pull the code from github.

If you use it, let me know what you think. I'm on Twitter: @jfeldstein

An Hour With Sean Ellis: Improve Conversions with Questions that Confirm Intent and Uncover Objections

I spent an hour with Sean last week, and he was all about asking questions that fix problems. That makes sense, given that his product, Qualaroo, is all about asking questions inside your app to better understand your users and remove friction app.

When we at first used Qualaroo on BuyAds.com, we asked broad questions to better understand what our audience was like (“Have you bought ads before?”), but the answers didn’t lead to direct action to improve our product or processes.

Sean was nice enough to spend time showing us a better way, including showing us how Qualaroo’s newest features line up with what we ought to be doing instead.

This won’t solve every problem with your product: If you’re wildly misfit to your market, or just aren’t close to a product people want, then you’ve got to get back out of the building. But if things are turning, and you want them to spin faster, this is a great tactic.

A Repeatable Process for Question-Driven Product Development:

1. Identify poorly converting areas of your product.

You already know how to do this. Peep your conversion funnel for steps with strong drop-offs, areas where you know what you want people to do but too few of them are doing it. These aren’t problem areas, so much as the areas where the most potential for growth reside.

2. Ask users open ended questions to learn their objections while they’re experiencing those holes.

At BuyAds, we have a lot of traffic sent to our catalog pages from specific publishers. These are people who self-select themselves as shoppers, should have been Product Qualified Leads, but actually convert less than our average leads.

We don’t know why. We have suspicions, maybe the users were just curious what’s available and never intended to buy, but guessing and asking users to confirm is shooting in the dark (which is what we did on our first survey). Better to ask open ended questions and listen to what your users have to say.

Compare shooting in the dark to the following survey made using Qualaroo’s branched survey tool:

Qualaroo branched survey soliciting user intentions and objections

Click for full size

The user drives the discussion, and tells us exactly what we want to know.

3. Collect and deal with common responses.

If everyone complains about the same thing, you know exactly what to fix, with confidence that you’ll see an uptick when you ship it.

 

Try Qualaroo. Talk to your users. Go nuts, and share what you find.

 

Measure mobile sdk usage and report per user stats in your Rails app, in 10 minutes, using Segment.io and Keen.io

The Background

I’ve been helping InfoMeters develop their on-boarding and user management flows. We realized things can go a lot faster if we got their development outside of Rails, so they could build the logic they wanted inside apps meant to deliver the same functionality, without needing to code everything by hand. Turns out that the growth / analytics world has tools ready for the job.

The Features

  • Track each API user’s usage of the app
  • Show each user a chart of their own usage over time

The Tools

Since we want to do this with as little coding as possible, we’ll lean on the following popular 3rd party tools:

  • Segment.IO fans data and events out to the relevant tools which each perform their own role. We’re only using one tool in this example, but, trust me, you want to integrate these guys first so other tools can be simply turned on/off when you decide you want to try them.
  • Keen.IO acts as the data warehouse, logging event and usage data and performing and visualizing all kinds of queries with their simple SDK.

1. Register for Segment.IO and Keen IO

Put your Keen IO secret key into your Segment.io integrations panel, and turn on the green slider.

2. Pick up your Segment.io secret key

It’s on their Ruby setup guide. You’ll need this in the next step.

3. Setup Segment.io into Rails

  1. Add Segment.IO to your Gemfile
    gem ‘analytics-ruby’
  2. Initialize Segment.IO from inside config/initializers/analytics-ruby.rb
    Analytics.init(secret: ‘[Your Segment.io Secret Key]‘)

4. Track important activity when users perform those actions

In your Rails code, wherever you want to track a special event:

You can also include other user attributes in the :properties hash. InfoMeters is using this approach, tracking each use of the SDK, passing up a client API key that’s known to both their web application and their Android/iOS SKDs, to combine per-client analytics reported from each environment. Keen will let us filter events for those matching properties (like a certain API key) which lets us look at users’ data individually, later.

5. Visualize each user’s usage on their dashboard (or any page you’d like)

Keen offers a visualization SDK (a javascript library) that builds nice looking charts from the data you’ve reported.

Unfortunately, it’s very complex. Luckily, friendly open-source folks at CultivateStudios built a wrapper around Keen’s SDK called Keentivate, which provides a very simple interface to the same code. Let’s use that. =^]

5.1 Download keentivate.min.js from the Keentivate repo to app/assets/javascripts/keentivate/keentivate.min.js

5.2 Initialize the library with the following javascript in app/assets/javascripts/keentivate/initialize.js: (Get your projectId and readKeys from your Keen.io account)

5.3 Add a manifest file (app/assets/javascript/keentivate/index.js) that will load these files in order:

5.4 And add to your main app/assets/javascript/application.js manifest (sigh, le Rails):

5.5 Now, where you want the chart to show up, add the right HTML:

The following chart should show a line graph of Used Feature X events, performed by the current user over time.

Special thanks to @calvinfo for helping me sort out some of the technical mashingups between Segment.io and Keen.io, @michellewetzler for introducing me to the idea of keen-powered per-user analytics, @Joe_Wegner for the Keentivate library and to the Keen.io and Segment.io teams for great products with amazing service.

 

The 1 question I asked my team that got us to Product Market Fit

What are we not doing right, yet?

Ask everyone on your team who talks directly to customers what it is they think you’re not doing right yet with your product, and put the responses together in your mind.

I was surprised which needs we’d been hearing that were suddenly forgotten about, and which quiet wants turned out to be the things everyone agreed we needed.

Go ahead, try it. Let me know if it changes any of your Product priorities.

 

Analyze Retention From Raw User Data

If you follow along with tools like Keen.io or MixPanel, you’d get the impression that calculating retention was a simple matter of just asking your analytics tool for retention numbers. Keen.io even wrote a very streamlined how-to in their blog, Radical Transparency: How to do a retention analysis (even including scripts you can use on top of your Keen account)

But what about when you don’t have pre-packaged analytics tools setup in advance, and still need answers to these questions? This was one problem I ran into while jumping into BuyAds.com.

This post explains how to calculate a retention analysis using only your app’s raw user data, without falling back on any 3rd party analytics tools. It’s not that hard. ;^]

Example User Retention Over Time

What is retention?

Retention is a measure of how well your app keeps its users over time, sort of the opposite of Churn. Improving your app’s retention means more people sticking around for longer, which makes it much easier to grow your user base and build up strong network effects.

A retention curve is a way of describing the rates at which users abandon your app over the span of their lifecycle, and usually captures the idea that most people are likely to leave right away, but, the longer someone does stick around, the more likely they are to stay even longer.

Defining what makes an active / retained user, in the context of your app:

Most consumer apps calculate retention and “active” users based on when they last logged into the app, but this isn’t a golden rule.  Your definition of “users retained” might differ from that of your peers, or the standard, consumer-app model. Do what makes sense.

For instance, when calculating the retention of BuyAds.com advertisers, we decided to measure “latest activity” in terms of when someone last placed a purchase, rather than when they last logged in, because purchases have a much stronger impact on revenue than log-ins and that’s what we’re trying to model in our case.

Query for users, when they joined and when they were last active.

Once you’ve defined your concepts, call up your raw data. My query for BuyAds looked like:

SELECT
user.email,
user.created_at,
MAX(order.created_at) as latest_purchase_at
FROM
user
LEFT JOIN
order ON order.buyer_id=user.id
WHERE
user.role=’buyer’
GROUP BY
user.id
;

This gives us the raw data we need to calculate how many of our users remain “alive” after each week of the average lifecycle.

Query your app for the relevant data, and save it to a CSV that we can post-process using the following Ruby script.

Raw CSV data describing the lifecycle lengths of my users.

Raw CSV data describing the lifecycle lengths of my users.

Crunching raw user data into a simple curve:

Once you’ve queried for the raw data, you’ll be able to compute the averages you need in order to see the patterns that exist.

The basic idea is to:

  1. Iterate from 1 to 52, call it X.
  2. Take a subset of your users who joined at least X weeks ago, count them.
  3. Count the number of users who were still active X weeks after joining.
  4. Store the resulting % of users who were retained after X weeks.
  5. Go back to #1, incrementing X.

Peep the ruby I used to perform this calculation: https://gist.github.com/jfeldstein/5479549

Now you have the data to paint retention curve.

You can graph it so it looks super sweet (like the top of this post).

Or you can work it into the math of your growth model for your app. Rahul Vohra wrote a solid 3-part walkthrough on assembling a full Excel model describing your app’s growth or decay over time, which I’d recommend to take this further.

 

Growth Engineering @ BuyAds.com

BuyAds.com Logo

BuyAds.com was launched by isocket.com last year, when it was the focus of the demand side of our business. As isocket changed its sales pitch to focus on white glove service, BuyAds, a long tail business, spent the next two quarters in decline.

I’ve recently been given the reins to BuyAds.com, with the directive to turn around that part of isocket’s business.

So far I’ve gotten a team together, modeled our understanding of what might drive growth in the product (it’s a marketplace product, so there’s a lot to consider), planned experiments meant to improve specific areas of that model and all of us have gotten our hands dirty executing those plans. The rest of my time, I’m measuring what we execute on, and sharing quantified positive impact with the rest of the company. We iterate on, or jump from plans that aren’t winning, and double down on what does.

It’s a brave new world, and I’m having a blast meeting my new peers, sharing stories of successes, fails and the culture shock of being on, or starting, a growth team inside an existing company with traction.

Virtues of a Slow Launch

Sometimes, I lose track of how long we’ve been haphazardly working on Sway.fm.

On the surface, it’s been a long time spent on a product that’s yet to take off. But it’s been time well spent:

We’ve been actively talking to users the whole time.

That’s a lot of email,  Twitter and, whenever possible, in person chats. If you’ve met me in real life, there’s a good chance I at some point ask you how you listen to music, probing for what apps you use, what habits you indulge  and how those products win or fail at satisfying your habits. These conversations with users add up to an understanding of which users we mean to target with Sway, how they want to interact with their music and the opportunity to satisfy where current products fall short.

We learned from those talks, and iterated often as we’ve learned.

By design, there’s yet to be a major, public launch of our product.

We build a lot, and test the product with friends, family and a few public figures whom we trust with access to our very alpha product concepts. Each time, we hear what they like, see what they use and ignore and what they continue to ask us to build.

Over time, we’ve honed in on the core use case we trust will catalyze a habit in our users and we’ve experimented with UI concepts, design variations and flows. Now, we’re close to something that resembles a marketable product, and we’re stoked enough to start readying to launch it to a wider (ie, public) audience. Get our existing Chrome extension if you want early access to that upcoming product.

We’ve ended up having built. A lot. Of tech.

My GitHub heatmap.

Suffice to say that our team of two engineers has experience with the latest music tech available. We’ve spent afternoons sharing desk space and hacking together with other great startup teams (music tech and beyond).

We have private github repos for iOS, Android, Spotify and Rdio apps. We have hacks built on top of the apis from Ex.fm, Rdio, YouTube and more. We have hacks for sites that don’t even have API’s (because we love going where we’re not supposed to be). Our massively parallel node.js scraping stack can pull in years of Pandora thumbs ups in just a few seconds. Want those thumbs ups synced with your rdio library? We have backend for that.

We’ve built hacks ad infinitum, because (we like to, and) we believe you never build a thing right the first time. As a result, we’ve built up two years of tech assets and neat side-projects. That’s a lot of first-times out of the way.

Now we know what we want and how to build it.

It’s taken a long time to get here. We’re excited to soon be launching hard, and hopefully to contribute to shaping the future of how you listen to, collect and explore music in this post-mp3 world.

Interested? Get in touch @swayfm.