8 votes

The dangers of A/B testing and "funnel hacking": How we made things worse by making them better

4 comments

  1. teaearlgraycold
    Link
    I’ve worked at startups when they’re doing funnel hacking. I think the main difference is we were trying to measure a difference in user behavior over the course of days, not months. If you...

    I’ve worked at startups when they’re doing funnel hacking. I think the main difference is we were trying to measure a difference in user behavior over the course of days, not months. If you increase signups to a program that won’t finish for months then you can’t call it a success until you’ve seen that play out entirely. The funnel hacking I saw play out would get you instant feedback on signups and a week of lag time for changes in conversion to a paid user.

    Separately there’s the author’s own takeaway - that funnel hacking was fundamentally at odds with their product. Funnel hacking is an attempt to open the gates wider and let in people who are less engaged. That works great when your funnel is a signup process for a free trial or freemium product. That’s not so great when you’re entering the user into a contract.

    6 votes
  2. Adys
    Link
    Editorialized title because this is a VERY interesting article and the original title doesn't convey what it's really about. Discussion on HN: https://news.ycombinator.com/item?id=33022717

    Editorialized title because this is a VERY interesting article and the original title doesn't convey what it's really about.

    Discussion on HN: https://news.ycombinator.com/item?id=33022717

    1 vote
  3. [2]
    riQQ
    Link
    Meta: posted to the wrong group, no?

    Meta: posted to the wrong group, no?

    1 vote
    1. Adys
      Link Parent
      Uh, yes, misclick, meant to post in ~tech.

      Uh, yes, misclick, meant to post in ~tech.

      1 vote