Why We Shut Down Charm on the Eve of Public Launch, at $48k/Year and Growing

This is part of my 4-year bootstrapping retrospective. Part 1: Why Bootstrapping Was The Only Logical Choice.

A couple months ago, I found something I wrote accidentally on Hacker News. Having vitriol spewed at me. (Surprise! Some might say those two phrases are redundant! ;)

Why?

My husband and I had shut down our second SaaS product, Charm.

Charm was beautiful & wonderful (with — dare I say it? — a revolutionary workflow) and our early-access customers loved it. We’d been working on it for two years. We hired an expert sys admin to design us a scaleable architecture, and we had arranged the servers. We’d had freelance help all along, and we paid for the best (and while the best loved us and gave us discounts, the best is never cheap!).

By October 2012, Charm was already covering its own (massive) server bills — or, in other words, bringing in the better part of $4,000 a month in revenue. That was before we even opened it up to more than just a tiny sliver of the 3,800 people who signed up to “Be the First In Line.”

While our first SaaS, Freckle, had just recently topped $400,000 in revenue a year, I was sure that Charm would grow ten times faster.

Then we killed it.

Why? Why would we shut down a product that our customers loved, that was already monthly net-neutral, that we’d spent 2 years and something like $175,000 to $200,000 on?

This is what Hacker News couldn’t understand.

I wrote an email to our beloved customers, explaining:

You probably noticed Charm had some nasty downtime a couple weeks ago.

Service quality is very important to us. If we didn’t think we could do better, we wouldn’t do it at all.

We’ve spent very generously on sysadmin services and infrastructure (nearly $100k of investment on sysadmin services/infrastructure alone). We hired the best possible, and we splurged on a redundant, powerful, and expensive server configuration from the beginning.

Now we’ve discovered that there’s some kind of base incompatibility with Ubuntu, which is giving us kernel panics which nobody can track down. Charm has been plagued by mystery technical problems from the beginning, when we had to backport from Rails 3.x to 2.x because of massive performance slowdowns which even Rails Core members couldn’t identify.

What this has really shown us is that, if we open Charm to the general public, we won’t be able to provide you with the kind of service you deserve. We are a tiny team, and so far, we’ve had zero luck in our attempts to grow by hiring developers. Problems which are small now will only get bigger.

There are a lot of things I’m willing to take risks with, but not with your ability to provide support to customers for your business.

And so it is with a very heavy heart that we will cease operating Charm from Dec 15, for the foreseeable future.

You won’t be billed again, and we’ll refund your last payments.

We will gladly help you migrate your data out of Charm. Please contact us directly ([email protected]) for help.

Thank you so much for taking a chance on us, and sharing our dream for a superior email interface.

I’m truly sorry to disappoint you.

Best wishes,

Amy

That’s what ended up on HN. And all Hacker News could see was the technical issues. But I’m sure you, dear reader, have better reading comprehension than that.

It wasn’t about the technical issues. They were just the harbinger.

We shut down Charm for the best of all possible reasons:

  • Feeling responsible to our customers.
  • Feeling responsible to our own values.

Our company is three people: Me, my husband Thomas Fuchs, and one employee, Devon.

We’d already spent the more than 1/3 the cost of our house on development. (Our own money, in case that wasn’t clear. You know my position on funding.)

So, by the time fall 2012 rolled around, we knew it was “Now or Never” for Charm’s public launch. Cue another list:

First of all, it had been a long time coming.

Secondly, I didn’t want to keep bank-rolling it from the money we made from other projects (namely my class, 30×500). There’s no point in borrowing from Peter to pay Paul forever. I’m running a business, not a playground.

So we set a date: November 2012. This impending launch clarified a lot of things.

First, it was clear that Charm was going to be a much more demanding product to run than Freckle Time Tracking. For starters, to get great performance to grow to hundreds of customers, we needed $3k/mo of servers. Charm dealt with email; that’s a lot of backend connectivity involved, not to mention a lot of data to store, index, access, and search. We also built a totally live interface using Backbone.js and that is not without its performance costs.

Secondly, Charm, unlike Freckle, required high availability. If your time tracker goes down, it doesn’t actually prevent you from doing your work. Charm, on the other hand would be people’s work. Charm downtime could possibly cause our customers to lose customers.

That’s an awful lot of responsibility for a technical team of two (one sick).

Thirdly, we had two products to run: Freckle, and Charm. Plus my class. We’d already trimmed our sails by stopping our JavaScript Master Class for months at a time. It was still too much. Charm wasn’t moving forwards as fast as we needed, development-wise. Freckle wasn’t moving forward at all. Something had to give.

Therefore, we knew we needed either to find the right employees right away (fat chance!), or find technical partners to help us run the app.

So we took the partnership route with people we knew, respected, and trusted.

It didn’t work out.

This wasn’t anyone’s fault. We’re still friends and they’re fantastic at what they do, and we still work happily with them. But it’s been my experience, again and again, that just because you work beautifully with someone in one capacity doesn’t mean another arrangement will necessarily work, for all kinds of reasons.

(This applies equally to all permutations of consultant->partner, friend->partner, friend->freelancer, friend->employee, freelancer->employee, employee->freelancer, etc. My husband and I work together like a well-oiled, if occasionally cranky, engine. But that wouldn’t work for most people, either.)

So, while all this was coming to a head, in October… we were on a trip. We’d intermittently had weird Charm crashes, etc., for which we relied on our aforementioned expert sysadmin (to the tune of $200/hr — well worth it, by the way).

We had been at a biz conference in Scottsdale, and then rented a house for a week in Sedona, AZ. Thomas, Devon and I put our heads together for several days to work on the Charm launch. Then Devon headed home and Thomas and I had a couple days to ourselves.

Charm went down one night at 3am.

Our freelance sysadmin wasn’t available.

The servers weren’t responding at all. It wasn’t just that the app / web server had lost the thread… the whole system was unreachable. Thomas busted his ass to figure out what was wrong. We had to rely on Rackspace’s excellent service to get things back online. Nobody did anything wrong, but it still sucked the big time. My husband gets real tetchy when he is stressed out. (As do I.) We were supposed to be relaxing. It didn’t happen.

We have friends who run infrastructure products… and they thrive on it, but during this minor catastrophe, we thought back to their lives, and we had flashes of the future, of receiving server alerts in the middle of a party, of having to scrounge up a laptop to fix it (or leave)… no thank you.

Above, I broke the timeline a little — it was about a week after we returned from this trip that we decided to end our partnership. Things simply came to a head. Expectations weren’t met, words were exchanged, tears were shed (at least by me!). Stress, stress, stress.

So, there we were with:

  • An unwillingness to spend more out-of-pocket
  • A failed partnership and no hiring prospects
  • No development momentum
  • Downtime, another mystery problem, after we had already had so many
  • A very real preview of what our lives would be like if we continued down this path
  • Customers who were currently very understanding, but who would no doubt cease to be, the less and worse we did
  • Another product with happy customers, easy to run, and profitable, which we were neglecting
  • Did I mention I have a chronic illness?

Our choice was clear… there was no choice. Charm was already dead. We just called off the chest compressions.

So…

Charm is by far the best thing I’ve ever designed. I love designing software and I believe I have a unique approach; workflow, and good feelings, above all else, are my focus, and whenever I start to question my own self-regard, I use other people’s software.

Charm really is great. When people use it, they love it. And they happily give us money. (Lots of it! Charm was expensive and that didn’t deter our early customers.)

But what good does all that do me if we’re miserable? Or if we have to sacrifice everything else we’ve built trying to shove this fat baby bird out of the nest?

So of course, I love the product and I want to see it out there. We thought, we can give my baby up for adoption, maybe. I talked to some people in the industry about selling it. (Another list!)

First, I knew another bootstrapped software company just like ours that attempted to sell a (profitable) product they needed to “twilight”, for the sake of focus. They loved their customers and wanted to ensure continued service. Were they able to sell the product? No. They contacted all their competitors and a lot of other folks besides. Nobody wanted to buy it without the team, or they just wanted to buy the customer list (and shut down the product).

I asked one of the many venture capitalists who reach out to me: OK, I’ll bite. How often do product-only sales happen? Just about never, he said.

I asked around the folks I knew who were knee-deep in the more traditional startup space, where acquisitions happen. No good prospects there, either.

So, continuing the way we were going: Hell no.

Selling: Not an option.

Emergency hiring: Not an option.

Partnerships: Been there, done that, ripped up the t-shirt.

Taking investment: Hahahah. I only included this for the humor value. I’d take out a second mortgage before I took investment.

Shutting down with dignity was the only option left.

And so I wrote that heartfelt email, and sent it, and our customers were all really nice about it. They knew we’d always striven to take great care of them, and people respond to that.

Finally, we helped some of them migrate their data out, gave them nearly 8 weeks to make other arrangements, and that was that.

Charm is now gone. The landing page is gone. We use it internally (on a slooooow server) and that’s that.

What have I learned?

Well, for one, making the right decision always makes you feel better afterwards. Or at least, it always makes me feel better. It sucked, but it was a huge relief.

Two, if you treat your customers right, they’ll treat you right. Unlike some services that get acquikilled, we didn’t just shut down and delete data without warning. We did the right thing… as much for their quality of service as anything else. And our customers respected us for it. The general response was, “Aww, we love Charm, but that’s so sad! But we understand.” Some of them even thanked us (!) for making the right, hard decision.

And friends asked, “Are you okay?” like it was my cat that died instead of my product. That was sweet and meant a lot to me.

Three, the loud-mouthed people “in the stands” at Hacker News are full of crap. But, no surprise there. “OMG HOW DID YOU SPEND $100K ON THAT?” “For that much, you could hire TWO sysadmins for $5k/mo.” “Nobody will ever fund you now!” “Nobody shuts down because of technical problems! There must be a conspiracy!” Lulz. Sound and fury signifying nothing, my friends.

(Don’t believe that people would really say those things? Here, go read for yourself.)

If you were to only see the HN thread, you wouldn’t know that the people who matter (our customers) were kind. I bring it up because I believe this kinda crap has a chilling effect and I like to show what it’s really like.

Those were the lessons about the shut-down itself. As for what I learned about the whole two-year process:

Just because a genuine need exists, and you can fill it, doesn’t mean you ought to. Yep, I can design a better support tool than just about anybody else before and, so far as I’ve seen, since. But that’s not good enough.

I hadn’t fully thought through the issues of running an infrastructure product. Freckle‘s a doll to run. Charm would be, by its very nature, much more demanding. If I had anticipated our difficulty in hiring-people-to-worry-about-it-for-me, I would never have embarked on the project at all.

It’s easy to assume you’ll grow by hiring, right up to the point where you actually experience hiring someone. Big “duh” there. Everybody talks about growing with bodies as if it’s straightforward, even if finding talent is a challenge. But what I’ve found is that there’s little problem finding people to hire, but hiring is such a risk because if you hire the wrong person, it can ruin everything until (and long after) you end the situation. Devon has absolutely been the ideal hire for us… but before Devon, I had hired two people I was forced to subsequently fire. The stress was terrible for my health. And my health is far more important than any kind of glory, growth, respect, or revenue. I cannot keep hiring if I keep a 33% hit rate. (And, if you’re hiring and you plan to be a kind of semi-absent owner (either due to putzing around Italy, or sick), finding the right person gets even harder.)

You can do everything right… and still have it blow up in your face. In terms of the tech, we knew what we were doing. You might say Thomas knows his way around Rails, as a Rails Core Alumnus, and we certainly know what’s what when it comes to JavaScript performance. We hired people who were top experts in their field (including Rails committers). When we had those unbearable, unmanageable Rails 3.x performance problems, members of the Rails Core team helped us to try and track them down. They couldn’t figure it out. When we had another issue with the server architecture, nobody could figure out what it was. We don’t blame our sys admin at all. He has an amazing track record and we know he did a great job (sometimes at his own expense because he, too, was frustrated with the situation).

There is nothing at all wrong with our former partners; we love them. We gladly work with them to this day. It was simply that the particular structure of relationship (partnership) wasn’t one that worked for us both.

You can be a world-class expert; you can hire world-class experts; you can do everything right… and things can still go wrong.

There is no certainty in this world. There is no protection. Sometimes there is nobody to blame.

Making decisions out of boredom is pathetic. I love a good challenge. My mother couldn’t afford to replace my ancient Centris 610 when I was an 11-year-old begging for a PowerPC. So what did I do? I made the money myself. I sold all my My Little Ponies and a bunch of my other toys at a yardsale; I did errands; I washed cars. I rise to challenge. That’s one thing that’s never changed about me.

This time, to my detriment. Charm arose partially out of rage at the shittiness of the existing products (which we had to use endure every day!), and a knowledge that there was an open space in the market, but it was also most tantalizing because Freckle, good ol’ Freckle, happy Freckle, was boring. I wanted a bigger challenge.

Duh.

Ego is stupid. We had a little bit of Second Product Syndrome, make no mistake. Launching Freckle was stressful, sure. But launching Charm was much more stressful. My actual thought process went something like this:

“OK, when we launched Freckle, we weren’t really risking anything reputation-wise. But now we have a reputation and it’s important to ship something that, even if it’s incomplete, compares favorably to what people expect from us.”

Who knows, possibly if we’d ignored that bit of stupidity, we’d have launched Charm as a Shitty First Draft, and found all this out sooner. Or perhaps attracted the perfect technical hires by serendipity & being “out there.” We’ll never know now, though, and that’s ok.

I’m smart and learn from my mistakes. By the time we shut down Charm, I had already seen, admitted, and internalized all of the above. I’ve got no problem at all admitting when I am wrong. (Psst: I was wrong.) And so I didn’t worry about how our “reputation” would suffer for shutting it down (what does that even mean?) and I didn’t really worry that our customers would hate us, because I knew we had given them no reason to.

We’ve doubled down on Freckle and it’s growing, we’re happier, our customers are happier, we’re still working with our former/would-be partners, and all is rosy in Hoy-Fuchsville.

That leads me to the present: I’m writing this essay to share what I learned with you.

Despite this all sounding like a horrible (and preventable) situation, we made the right (hard) decision, and in my opinion, I’ve come out looking feeling good about my husband and I and the way we handled things, and still have the respect and friendship of our now-former customers and partners… and that’s all that really matters to me, in the end. What are the fevered dreams of random strangers on the internet, compared to that? Nothing.

Get my next bootstrappy gettin-shit-done essay delivered straight to your inbox. (And be first in line for tickets & discounts.) Drop your name in the box!

Discussion

  1. Brian Morearty

    Hi Amy,

    Twice in this post you used the term “infrastructure product.” Can you describe what you mean by that? Do you mean a product that requires 24×7 uptime? Or one that is core to keeping someone else’s business running? Or something else?

    Here’s the second reference:

    I hadn’t fully thought through the issues of running an infrastructure product. Freckle‘s a doll to run. Charm would be, by its very nature, much more demanding. That’s not a mistake I’ll make again.

    Thank you.

    Reply
    • Amy

      Brian, I think of “infrastructure product” as being something that other people must rely on to achieve their work goals. A time tracker for example, is a side tool. You can do your work without. Customer support, deployment tools, email/communication services, etc., are infrastructure.

      Reply
  2. Naomi Niles

    I think you did a brave thing. In the end, people won’t remember the “failure”, but instead your honesty and integrity in handling the situation.

    Kudos. :)

    Reply
  3. Wouter
    1. You seem somewhat obsessed with HN. You mention it pretty much every day on Twitter. If it frustrates you so much, why bother really??!!

    2. You sometimes share that you have some sort of chronic disease, but I have never seen you say more than that. Would you / could you share more about it? Or is that too personal.

    3. You also seem pretty obsessed with the concept of ‘your own home’. Every day there’s a few tweets on some sort of furniture that’s ugly / expensive / awesome. I have no opinion on that, just something I noticed :-).

    Reply
    • Amy

      Hacker News is a great source of amusement for me… and for converts. There is a huge percentage of quiet lurkers who are both A) nice people and B) interested in non-standard viewpoints. (Which weirdly includes bootstrapping and generating revenue.) Also: No point in preaching to the choir.

      I have Chronic Fatigue Syndrome, which isn’t a disease but cluster of symptoms that can have a complex cause. In my case, I had recurring Mono. I don’t like to dwell.

      We just bought our first home and I love it. I love design of all kinds, furniture is one I only recently have room for!

      Reply
      • Amy

        Assuming your comment was meant in a supportive way, I have lost a lot of weight and it has helped a little… certainly being overweight can make you very tired but it doesn’t cause increased viral titers, 103° F fevers, low body temperature, salt loss, post-exertional malaise, postural orthostatic tachycardia syndrome, so on & so forth, which are among my many symptoms, all with a definitive and immediate last-week-I-was-just-fine-next-week-I-was-basically-disabled onset.

        A lot of people who are having a genuine and immediate medical problem (like me!) get told by their doctors “Of course you can’t stand up without feeling like you’re going to pass out. You’re fat. You’re just ‘unfit.'” even when they could do that without any issues at all, just the week before. Which is a total farce and leads to some people suffering forever without getting help. Me, I accidentally found out it was due to the mononucleosis, and was able to get treatment, no thanks to the doctors who only saw my pants size.

      • Pier-Olivier Thibault

        I don’t know nothing about you and neither do I know anything about CFS so I don’t claim to have a solution for you.

        However, I came across this video a few years back[http://www.youtube.com/watch?v=dBnniua6-oM] and it literrally changed my perception about food.

        Maybe it can help you too, who knows.

  4. scribu

    At the risk of sounding like a spambot, this feels like one of the most insightful posts I’ve read on this blog and in general.

    Thanks for sharing!

    Reply
  5. Doug

    Hi Amy,

    It was quite refreshing to read this, written by someone who knows what they are doing. A client of mine has been skating by for 4 years now, constantly 3 months late on invoices, I’m the entire IT staff (contractor), and it takes 3-6 months to acquire a new customer for this business as it is a B2B enterprise type SaaS thing. Or it could be, had they done what I suggested 18 months ago. But they thought I was just trying to drum up more work for myself… but I was looking down the road for when the tech problems got harder and harder. i.e. make my job ultimately easier / cheaper.

    Long story short, the fundamentals of the business are unsound, and recent ethical lapses have made me realize there is no long term success here. The business itself was a very good idea, which is why I’ve stuck with it so long. But the execution… I did my best, but there is only so much one lone developer can do.

    So to read your story, where the execution sounds like it was done right, but you ran up against a brick wall and made the correct (and hard) decision is gratifying. There is a lot more to “IT business” than the tech centric crowd would believe – solving the tech issues are usually easy, it is the non-tangible and the people issues that aren’t. Business is more than just putting up a website and making cool tools to play with.

    Reply
    • Amy

      You can’t help people who don’t want to be helped, Doug. And yeah. The tech is the easy part.

      Reply
  6. Zach Russell

    Amy, It really sucks that you had to shut down charm, from what I hear it was your baby and users really liked it. It’s good that it was shut down with dignity, I have had experiences with other software that the providers will milk the software to the very end, no matter how bad the service.

     I'm sure the next think you and Thomas create will be amazing and I'm looking forward to seeing it :)
    

    Zach

    Reply
    • Amy

      Thanks, Zach. We have seen those other, “milked” products you mentioned, too. I would never perpetrate that on my innocent customers.

      Reply
  7. Angel

    I have a lot of respect for the integrity that you, and the rest of team Freckle, show in everything you do. I just wanted to say thanks.

    Reply
    • Amy

      That page you linked to, that had I linked to, was basically it. There was a PDF user guide but it wasn’t amazing.

      Reply
  8. Thibaut Assus

    Hello Amy !

    First of all, as always, you are really brave, and what I like is that you share everything. I think someone should write a book on you or create a movie :)

    You say it’s an infrastructure product, and I think you are totally right with that thought, but in your options, you never talked about open sourcing it, and let people run it themselves !

    As you kill it, I think customers could run it themselves ! And possibly after a while, you could find THE good commiter to github.com/slash7/Charm.

    You know, a lot of artists are destroying their paintings to be separated of them, and to start fresh !

    What you are doing is not so different than what Gainsbourg (a famous french singer who started by painting) did !

    But please, let the community decide about that, I think Charm deserve that, at least :)

    Reply
  9. kr1t1c4l

    Hi Amy Thanks for writing about your experience and don’t beat yourself up about it. At least you tried and you had the courage to kill it off. It is hard to know what is minimal in “minimal viable product”. Did you consider the option of open sourcing your system? That may allow someone else who doesn’t mind the rigors of infrastructure support to run it. Alternatively, it may allow your old customers to run it in house. Just seems a waste to leave all that effort sitting on an in house server. The shuttering of Wesabe was an example of how to do it – they attracted some developer interest when they announced they were closing down and open sourcing their core product.

    Reply
    • Amy

      We’re not interested in OSSing it for a number of reasons:

      One reason is that if we serendipitously meet the right dev/hiring manager, we will hire them and revive the project. Secondly, as all bigcos who randomly outsourced a dead project can tell you, OSSing a huge project solves no real problems. Thirdly, neither of us will run an OSS project for it, and fourthly, it’s a bit of a beast. You can’t just pop Charm on heroku and expect it to run at speed.

      Reply
  10. Nate Berkopec

    Hey, I hate Hacker News commenters just as much as anybody else, but isn’t sending an email saying “we’re shutting down for technical reasons” then chiding HN because they thought you shut down for technical reasons a little, uh, weird?

    Reading your email, how is anyone supposed to infer from that that you shut down for mostly non-technical reasons? I mean, that’s a fine reason (and more difficult to own up to, so props for doing that here), but playing holier-than-HN in this case seems unwarranted.

    Reply
    • Amy

      Er… you don’t have to infer anything.

      Let me quote:

      What this has really shown us is that, if we open Charm to the general public, we won’t be able to provide you with the kind of service you deserve. We are a tiny team, and so far, we’ve had zero luck in our attempts to grow by hiring developers. Problems which are small now will only get bigger.

      Right there, in black & white, I say: “It’s a service quality commitment issue… we can’t fulfill our promise. We can’t grow by hiring. We can’t scale.” No need to infer.

      Reply
  11. David

    I’m just curious about the performance issues. Did you ever actually look at the server yourself, or was it just your sysadmins telling you that you needed a more powerful server? It does seem a little odd that you would have trouble handling a few customers on $3000/month of servers, although I’m not terribly familiar with what exactly your product did.

    I also understand about CFS, having dealt with it myself, so I know that your illness probably played a large part in your decision to shut down the service (I was in a similar situation myself 14 years ago). CFS tends to affect overachievers – we seem to have the capacity to work incredibly hard, which then burns us out.

    Reply
    • Amy

      It wasn’t a performance issue. Nobody was telling us “You need faster servers.” We invested heavily so we would be able to grow without having to upgrade mid-year, and a redundant architecture was important to us.

      Reply
  12. Dustin

    What about charging for it but not as SaaS? I.e., the customer installs it on their own server, and, if they need 4 9’s of uptime, hires their own pager-carrier.

    Reply
  13. Chris

    Sorry to hear about your problems with your app. That’s really sad to see you had to pull the plug, but i totally ‘get it’. Still, feel a bit sorry for you.

    Best of luck with freckle, i hope it does really well with renewed focus.

    This is really scaring me actually, i’m just on the verge of starting building a saas app in rails, i’m considering maybe using another platform now…

    Reply
  14. Henrik

    Very interesting, thanks for sharing. Knowing when to press forward and when to walk away is important but obviously very difficult.

    Also, what’s with this “Craig” asshole?

    Reply
  15. Patrick

    Thanks for sharing this. My highlights:

    • “It’s easy to assume you’ll grow by hiring, right up to the point where you actually experience hiring someone”

    Somehow ‘hiring’ is also part of the VC game if I am right. Good to hear that there are other options, to learn, and re-start so to say.

    Reply
  16. John Ward

    This was an really good and honest post about why you decided to shut down. It really clears up all of the HN assumptions. I’ve worked jobs where I was responsible for “mission critical”, or as you call it”infrastructure”, applications and the stress is completely ridiculous. Worst 3 years of my life! I’ve identified early on that I probably don’t won’t to run this type of product.

    I also like that you touched on the ego issue. Sometimes it’s hard to get over boring success. After awhile everything gets old and as people we seek out anything more challenging and engaging. It’s like every job I’ve ever had. Once the shiny newness wears off, then it’s just another boring thing I have to do every day. Not letting boring success cloud my judgement is something I will take away fromt his post.

    Reply
  17. FlyingAvatar

    Just curious if you considered open-sourcing what you have produced so far?

    It seems a shame to let something with the potential for success go to waste.

    Reply
  18. David

    I had a look at your product page on archive.org, and it does seem like a great product. It would be a huge shame to completely ditch the product due to some teething troubles. Can you not just use Charm in-house until you work out the kinks, before rolling it out again?

    PS, your system doesn’t send an email when someone replies to one of these comments.

    Reply
  19. Mark Roseman

    Amy, thanks so much for honestly sharing your experiences letting go of something you obviously care very deeply about. I think your description of how you managed the relationship aspects of your various partners is something not talked about enough.

    I’m also very glad you talked about the personal stress and health aspects of this decision. As I’ve said before, it’s been evident you’ve been dealing with a chronic illness for a while, yet what you’re able to accomplish is nothing short of amazing.

    To have the freedom to make these difficult business decisions by prioritizing very personal desires (product quality) and personal needs (health and relationships) goes to the very core of what not accepting outside money is all about.

    I know it sounds weird to say this, but a big congratulations are definitely in order.

    Reply
    • Amy

      Thanks, Mark!

      To have the freedom to make these difficult business decisions by prioritizing very personal desires (product quality) and personal needs (health and relationships) goes to the very core of what not accepting outside money is all about.

      A thousand times yes. Can’t tell you how many times during this situation my husband & I have looked at each other and said, “Thank god we have total ownership.” I’d spend that money again any day, if the alternative was to go back and do it with investment. My husband and I are of one mind on these big decisions. The very last thing we need is somebody else to appease, flatter, handle, or merely convince.

      Reply
  20. Sasmito Adibowo

    I’d love to know what were the technical issues that causes server slowdowns and other scalability issues. Was it Rails? Backbone.js ? Other stuff? Just for the sake of learning and not repeating the same mistakes that you’ve made.

    I’ve been doing desktop/mobile apps so far and am looking to add in SaaS components to the offering. Still on the fences on what backend technology stack to choose.

    Reply
    • Thomas Fuchs

      Hi Sasmito, as for technical issues, we where plagued by various problems that are not related.

      The app as it is, is using Backbone.js, Underscore.js and Zepto on the front-end, and Rails 2.3, Postgres, memcached, redis, resque, and for websockets Sinatra, and a few other things. The front-end is communicating with the back-end via a JSON API.

      I’ve come to the realization that this much client-side processing and decoupling is detrimental to both the speed of development, and application performance (a ton of JavaScript has to be loaded and evaluated each time you fire up the app). It’s better to let the server handle HTML rendering and minimize the use of JavaScript on the client. You can still have fast and highly interactive applications, as the new Basecamp shows—letting the server handle most stuff doesn’t mean that you have to cut back on cool front-end features and user friendliness.

      I’d argue that all these newfangled libraries are actually detrimental to the user experience in some ways, as they lock you into certain patterns (it’s hard do to things the authors didn’t anticipate) and if you use something like Ember (which we didn’t), it’s even worse as all applications using it practically look the same.

      We’ve spend a lot of time getting Backbone to work properly, as the easy-of-use quickly deteriorates when your models get more complex. It’s a great choice for simple stuff, but email is far from simple. We also had to add yet an other extra layer of processing to generate “ViewModels” on the server because the normal Rails serialization of objects wouldn’t cut it.

      What you end up with is building a layer cake that doesn’t add any value and slows down development. Especially when you’re starting out and need to stay flexible you don’t want to have too much code around—and Rails is great for that. But… adding a JSON API layer and basically a second application that runs on the client is annihilating this advantage for you.

      All in all, my current recommendation for SaaS-type web apps is: Rails 2.3 (or Rails 3.2 if you prefer), a Postgres database, as much HTML generation on the server as possible and augment that with RJS (Rails’ mechanism to push JavaScript snippets to the client that get eval’d). This allows for direct re-use of server-side templates, and it’s simple, and works well. As an added bonus, keeping things on the server allow for much better error and performance monitoring and thus quicker turnaround for fixes. There’s also a lot of great stuff in this direction coming up in Rails 4 (like Turbolinks).

      Alas, keep it simple and don’t repeat yourself.

      Reply
      • Babak Morshedizadeh

        I can relate directly to much of what Thomas says. I deal with this every day, at least, I have since 1998. I agree with you that server side processing to generate the DOM is the way to go, and client side is best left for “cool” effects (which may mean some DOM manipulation, maybe inserting some elements as data is loaded et cetera via Ajax et cetera. The only time I was able to build highly complex, very large scale, responsive, and massively client side applications for the browser was when I used GWT. What GWT did (latest versions) was give me the ability to look at the development process from a different aspect, not the pure HTML aspect, but a normal client side app. It also does more than just compiling the code into Javascript, and I am talking about beyond asymmetric transformation of Java to Javascript. You do need to still watch out for performance issues, that’s a given, but it does provide mechanisms in the code to easily work around it. The only issue: mashup is difficult. I was able to get much of that done, to have Javascript call GWT code, GWT call Javascript, all of it mostly seamlessly. But it was a “ton” of work. Using bootstrap I was able to make things look nice. Above all, GWT gave me “backbone” without giving me “backbone.js” and it did it in a very intuitive way. Google, as it tends to do with good things, somehow abandoned it and went for Dart. I am hoping Dart will be do better than GWT and it will stay around. The problem is, 99% of the web is pre Javascript (client side). So, since I haven’t found any viable alternatives, for now, I am looking at Python et al.

      • Noah Davis

        I agree generally with this sentiment. I’ve seen a couple cases where the choice to use a client-side framework early seemed to inhibit development speed. It’s important to stay relatively up to date with “newfangled” development frameworks like Ember, and it’s hard to do that if 1) you’re a small team and 2) you’re trying to keep up to date with security patches and the like on the server side.

        One minor point of disagreement: “if you use something like Ember (which we didn’t), it’s even worse as all applications using it practically look the same” – I haven’t found this to be the case. Ember doesn’t ship with any kind of pre-defined views. It’s strength is in keeping data in sync between various components, so I’d be surprised to find that applications look the same unless there’s cargo culting going on, which is really unrelated to technology.

      • Josh Szmajda

        Hey Thomas, as someone who’s evaulating a number of front-end technologies right now, I’m curious to understand better a couple assertions you made.

        “if you use something like Ember [...], it’s even worse as all applications using it practically look the same”

        How does this happen?

        Also you mentioned adding an “other extra layer of processing to generate “ViewModels””. Do you think that was a consequence of using Backbone specifically or do you think we’d have this trouble everywhere? Is there in your opinion a way to get around that?

        Thanks!

      • Thomas Fuchs

        As for your question on ViewModels—if you do any non-trivial resources, you’ll quickly end up with JSON objects that are just too large, especially for lists. For emails, imagine that you have nested threads, user avatar images, nested assigned cases, etc.

        Because of this, you’ll need specialized JSON objects/arrays for different use cases (search, list view, detail view, and others). It follows that you’ll end up with this with more or less any front-end framework (if you care about performance!). Doing this adds complexity, which can be avoided by rendering HTML on the server where access to arbitrarily deeply nested data is relatively cheap (and can be highly optimized by keeping snippets of HTML around in memcache, etc).

        Of course, you can optimize any system—my point is that for most web apps, simpler, server-side generation of HTML is a better architectural choice—less code (which means likely fewer bugs), easier testing and great performance if done right. At the same time, you can do parts of your web app with a client-side MVC if you need highly complex interaction (I found it helpful for real-time updates, for example). If you think it’s better, then you have a lot of choice—from rolling your own (which is pretty simple, actually) to relatively simple “pre-fab” solutions like Backbone, to more complex stuff.

        If there’s something I’ve learned in 25 years of programming, it’s keep it simple. Don’t add stuff and layer upon layer just because all the cool kids are saying you need it. That is all.

      • Mark Roseman

        Hell yeah. Sometimes adding the right layer can be a huge win (and often times that layer is something small, custom, and app-specific), but too many people are quick to absorb the (often very focused) benefit of adding a large generic framework without adequately assessing both initial and ongoing costs.

      • Thomas Fuchs

        Thank you, Tim. By the way, I’m not saying the client-side MVC is bad—it’s just that in my opinion it’s not a silver bullet, and for most apps it’s not a good choice for the reasons I’ve given.

        I hear from a lot of developers and companies I highly respect that they had great success with a hybrid approach, where they implemented some of their more interactive functionality with some form of client side MVC framework, but having most of the app using just normal server-side HTML generation. This keeps complexity down, load times fast and you can add the extra functionality (and extra complexity) just when you really need it.

      • Thomas Fuchs

        Hi Stefan! I’ve seen it, but I don’t personally agree with your approach.

        I think most web apps are first and foremost written for human beings to be used via a browser. An API (restful or not) is, in my opinion, not an integral part of that, but rather an add-on.

        There are various reasons why I think so, and I unfortunately right know I don’t have the time or inclination to write a lengthy comment about that. I’ll try to get a longer post together on this when I can, with examples of what I mean.

        This doesn’t mean it can’t work for other people—but for the way I design and create applications it’s not a good fit.

      • Stefan Tilkov

        Don’t want to hijack this discussion, but at someone I’d be interested in finding out how we created the impression ROCA is about APIs. Humans sitting in front of a browser is exactly what it’s about.

      • Thomas Fuchs

        I disagree with the philosophy that the URLs have to be restful (I still like them readable and “making sense” tho), that only “semantic” markup is generated and that it has to work without JavaScript. I find artificial restrictions like these detrimental to build great user interfaces.

        Content, presentation and behavior are things that work together and are integral to each other—and there are no “hard borders” between these. This is true for various reasons, starting from providing great, hassle-less user interfaces all the way to technical cross-browser issues.

        In my experience, it’s much easier to provide a separate API for machines—so you can fully concentrate on human beings and web browsers only for your main app.

  21. Sagar

    I don’t see anything wrong with thick client and thin server model. The Server is meant to do heavy processing such as accessing database, business logic etc,. I understand that having Backbone or any such framework can be bit difficult initially, but once learned and applied a pattern on the client side (not necessarily MVC) can help managing the application in a better way. Also, the fact that the HTML, CSS and JS can be cached by the browser and can be hosted on a CDN improves the performance of you web app.

    Reply
  22. Paul Biggar

    Curious about whether you looked at getting another team to take over? With so many startups today, it seems there would be many young, hungry teams eager to take over a much-needed product like Charm? I’m curious if this sort of arrangement can be made work, and if you guys have tried it or something similar?

    Reply
    • Amy

      Paul, did you read the article? Did you read what I said about partnerships? Assuming you did, lemme just ask… Why would I look for “another team to take over”?

      Reply
  23. David

    Couple of comments from me, also long time software business owner… I built my business from ground up, started with $500 investment to buy PC…

    It feels wonderful when you are finally in position to pay someone to do work for you. It gives you really good feeling, frees up your time and makes you feel in control and like you have some power… It starts feeling like you can tackle anything.

    But, your mistake was that you paid people to do something that you should have first done yourself. Paying someone to do work for you, but you not knowing how to do that work, leaves you exposed and without crucial information and most importantly understanding about what is going on inside.

    This is not problem if you have money to keep paying and replacing people that are incapable and until you find people that can get work done. But, if you do not have that kind of budget, what happened in your case is almost inevitable.

    If you did all the work yourself you would have learned ins and outs on what to do and what not to do and you would have either pulled the plug much sooner or, more likely, would not have been in the position to pull the plug at all. Nobody, not even at $200/hr, cares or is more invested in your business than you.

    I am not really chiding you, I am just trying to point out important lesson here.

    Hope this helps…

    Reply
    • Amy

      Hi David, I’m not gonna be too harsh on you because you’re repeating advice I gave to somebody else last week (reported in my last blog post)… except, of course, I asked questions to verify his background and skill set before telling him that, and you didn’t. You clearly aren’t aware of my or my husband’s background, or what we were doing on the project and didn’t care to check, either, before dispensing your hearty “$500 PC”-based advice.

      But then again, perhaps you really do practice what you preach, and do your own lawyering and doctoring as well.

      Reply
  24. Jess Greene

    This is such a great post. Thank you for sharing your story and your insights into running a startup with a conscience. I really appreciate hearing your perspective while being in the thick of it myself. Cheers!

    Reply
  25. Laurie Wheeler

    Thank you for this post, you have no idea how timely it is! I certainly don’t have the same scenario, but as a fuzzy end if a shoelace boot strapper, I have cycled to the end of my mission. I’ve had bizzaro land tech hiccups for a year, things that lead devs scratching their heads. I run solo, I outsource what I can (which isn’t a lot) and I’m super proud of what I have managed to do. I took a joke and made it real & valuable to my members. But I just can’t deliver the way I want to.

    I’ve spent a week agonizing over this, whereas I don’t have to give up the whole enchilada, I need to vastly restructure for everyone’s sake, including my mental health :) I have learned so much= win, but the whip & dead horse scenario has gotta go.

    Back to the purpose of your post. I think if a person has an ethical bone in their body, loves to create useful products &/or services, then we can say “ya know this just ain’t workin'” it’s not giving up if it doesn’t work. That is the single most important take away I’ve had all week. Thanks for sharing your experience.

    Reply
  26. Mark

    This article captures an often muted part of the narrative of entrepreneurship: the dignified exit. My own experience starting a college radio station had fleeting moments of succes surrounded by tumult and politics but the lesson of walking away in a dignified manner was more instructive than our groups minor successes. Being able to balance your customers needs against your ow is so hard and you and your team did an exemplar job.

    Amy, you have such a fantastic voice in your writing and a real knack for insightful polite discussion on the web. Cheers for keeping it classy!

    Reply
    • Amy Hoy

      Thanks, Mark!

      Part of my mission is to reveal the stuff that people won’t talk about so it was so nice for you to write that, I’m feeling pretty good about achieving my goals right now!

      Reply
    • Amy Hoy

      Thanks, Mark! :) I’m not sure anyone’s ever called me “classy” before… but I’ll keep trying ;)

      Reply
  27. Khuram

    Hi Amy,

    Could you try to recreate it in another language like Python or PHP? Since you said Ruby And Ubuntu were at fault so maybe a more conservative setup is in order?

    Scary read though.

    Good luck.

    Reply
    • Thomas Fuchs

      If you read the article again, we didn’t shut Charm down because of specific tech issues. We shut it down because we don’t want to run an infrastructure product.

      The complexity of software like this is very high, regardless of what you choose to implement it with. If it’s not Ubuntu kernel panics, it may be PHP security issues, or Apache segfaults, or Rails security hotfixes, or outages of external services, or issues with your provider’s network connectivity, or a fire in your server room, or a DOS attack against your servers, or a Python vulnerability.

      You can’t fix us not wanting to have the responsibility for a life-dictating service (“Can we go on this plane ride together? We’ll be offline for 8 hours!”) by swapping out the technology. There’s nothing that’s 100% reliable.

      Reply
      • ford.prefect

        And how about Freckle? Isn’t it an infrastructure product?

    • Brandon

      Working on an infrastructure product myself, I can tell you that no language, tool or technology will provide 100% uptime. Someone once said that any sufficiently complex system is in some sort of failure mode at any given time.

      I don’t fault them for shutting it down. Some people want that kind of life. Some don’t. To each their own.

      Reply
  28. Brad Oyler

    Hey Amy, Curious as to how you would approach the infrastructure now that you went through this ordeal. Anything you would of done differently? Thanks for sharing.

    Reply
  29. Hawk

    Hi Amy,

    Been a follower of you guys on Twitter for sometime, this post is very inspiring to read. Not just the mistakes or lesson & learn but heart & soul / blood & sweat experience. I hope my wife and I would go through similar stages like you guys did. It would have been totally worth it. Money is no biggie…

    Anyhow, I am coming from an Enterprise B2B background so I was a bit confused about the term “infrastructure product” until I read further down in the comments. I guess in my world, it’s basically called “Tier 1″ or “Tier 2″ Applications (the “mission critical” type). And I am familiar with various SMBs to Enterprises facing problems with application availability, how to handle failover, what to get for 4 9s or even 5 9s… Now these are expensive things to have!! However, there are some software out there in the market that can help applications to achieve near high availability using existing public cloud providers (AWS, Azure, etc…). I am wondering if you guys did some evaluation on 3rd party technologies that can measure, monitor, troubleshoot,… to ensure those Service Levels are maintained?

    Reply
  30. Ryan

    Thanks for posting. There are lessons here that can help us all and you are brave for acting on the decision and being honest with your customers, and also sharing it on the interwebs. Blessings on your current and next endeavors

    Reply
  31. Philippe Monnet

    Amy and Thomas, first of all I have been a huge fan of both for a long time. Thanks so much for sharing your experience in such details. It is super useful. Thinking about my day job, I realize that we have a super qualified infrastructure team running our data centers. Although we all live with the commitment to jump on Critical (System Down issues), that our pressure is very well spread out and shared. But for a small team I can only imagine how hard things would be. So good luck with Freckle and the next “amazing” product I know you guys will soon dream up! :-)

    Reply
  32. Bala Paranj

    I admire your honesty. Wealth is not just about accumulating money. Health is also very important. Have you heard about the book “Eat Right for Your Blood Type”? It really helps in increasing energy.

    Reply

Leave a Reply

You can use Markdown in this comment box.