Ruby

@RTR_tech

As I write this, today is beach day @RenttheRunway (July 31, 2014).

What does that mean? It means our CEO Jenn Hyman (@Jenn_RTR), decided that the whole team deserved a day of “Mandatory Fun” at an awesome beach with tennis courts and a pool and the whole nine yards, on the company's dime.

A few days before that Jenn spontaneously ordered an ice cream truck to come to the office and give everyone free ice cream. We're not talking 10 people here - we’re talking about the whole, enough-to-fill-multiple-busses, RTR office.

A few days before that there was a Hawaiian style BBQ/Luau at our Dream Fulfillment Center in Jersey (where your dresses are shipped from).

You get the idea.

But lots of startups have occasional crazy fun surprises like that (right?). What's really telling about a company is the day-to-day. This blog is about life @RTR_tech, not RTR in general. Make no mistake: this is a tech organization and it’s an awesome one.

Let me tell you a story. It's going to be a bit scattered, and probably long-winded, but I think you'll like it.

I promise I'll mention some tech along the way.

A little more than a year ago, I was living with my family in Ann Arbor, MI where I had worked for about 13 years. Ann Arbor is an awesome little college town, but my wife wanted to move back to NY where she grew up. As a prerequisite, I wanted to make sure I found a job in NYC where I could truly be happy. This was more difficult than you might imagine.

I was looking for a small-ish, startup-ish, flexible, open-minded tech company with outstanding people.

I eventually discovered a large list of NYC companies of this ilk. Cruising down the list in alphabetical order, I eventually came to Rent the Runway.

"A fashion/e-commerce company?," I thought, "meh."  I passed right over it.

Fortunately, it didn't end there. The CTO of another company (whose selfless kindness I should have repaid long ago) reconnected me with RTR_tech via CTO Camille Fournier (@skamille) I had a phone call with her and was extremely impressed. The rest is history. I also spoke to Vijay (@vjsubr), head of Analytics, Jenn Hyman and various awesome members of the tech team and I was convinced.

One year later, I can tell you that this company is far from just a "fashion/e-commerce company" and is anything but "meh."

These people are awesome and humble. They learn from their mistakes and they learn fast. They actively seek feedback. They listen deeply. They really, honestly care about their teammates. They have created a culture of openness, idea sharing, risk-taking, fun, friendliness, honesty, continuous improvement, teaching and in-the-end, success. And they pay me to hang out with them all day!

What follows is just some of the awesome stuff I’ve had the pleasure of living with @RTR_tech.

First off, we use modern technology tempered with a good bit of wisdom that the “Shiny New Thing” is not always as awesome as it seems. The grass is not always greener on the other side. We’ve adopted a lean-and-mean, modern Java web service framework called Dropwizard. Not Node. Not Scala. Not Python. They're all cool, don't get me wrong, but Java is tried and true and way cooler than you might think if you just take the time to get to know it. :) Plus: Java 8! And cool stuff like @AutoValue. We're working hard to move our front end code to Backbone. There's some Ruby in the mix for good measure. Plus, have you seen what renttherunway.com looks like? It's freaking beautiful! We work with awesome designers and we really care about making it not just work right, but also look great.

We have regular, weekly one-on-one meetings with managers, team leads and peers. These are not perfunctory tech status updates. In these meetings we talk openly about what's bugging us. We refine our goals and take steps towards achieving them. We work to carve out time for people to work on that open source contribution they've had in mind (making @AutoValue work with Hibernate perhaps). As a manager, one of my personal rules of thumb is that performance reviews should not contain any negative surprises (or only good surprises). I've learned to use this one-on-one time to make sure that's the case and that I'm guiding/teaching (and learning from!) my reports to avoid those negative surprises. Just a few days ago I was writing a performance review. When I really got into the meat of it, I realized that I had some new suggestions around goals the person could set, but that we hadn't talked about yet. Before I wrote those things into the review, I grabbed the person and talked through the suggestions. The next day, we had the Important Official Mid-Year Review meeting. I delivered my write-up, and watched the pride swell up in the individual. This is not to say there was no constructive feedback, but that that feedback had already been well established and internalized. That's how you do performance reviews. We do that here @RTR_tech. If you're on my team, you'll get a performance review like that. I really care - we really care - about our teammates as people. In the end, that's all we have. Java code is easy. People are hard. Help them refine and follow their goals. Help them thrive. Help them learn. Help us learn.

@RTR_tech we are a learning organization from top to bottom. As such we’re encouraged (by @skamille and others) to attend technical conferences. RTR_tech will send you to speak or just attend. And of course, @skamille, @markwunsch, and @OMGannaks held a panel discussion @RTR_teach - oops, @RTR_tech - about how to write successful conference presentation proposals, because… they're awesome.

Similarly, we do Drinks and Demos every Friday. This is our chance to share essentially whatever we want as engineers. Maybe it's @timx showing off the new RTR Unlimited product/feature his team just released or @ericqweinstein teaching us some new stuff about "The Rubies." It could be @CarloBarbara doing a case study of how to systematically debug a tough OutOfMemoryError, because, as @skamille says, "don't flail."

Speaking of "don't flail:" you can learn that and so much more by going to @skamille's weekly office hours and having a great discussion with her. I'm not exaggerating that every time you sit down to talk with her, you'll walk away having learned something or with a new perspective on something. She maintains these office hours as a way to remain accessible and keep the communication lines open despite the fact that our tech team is quite large and growing rapidly. How accessible? Our awesome intern Maude (@QcMaude - A Post Internship Look at RTR) has mentioned to me how great it is that she is able to openly talk to Camille as she would any other peer.

There are lots of big brains out there in the world though, and we can't send the entire tech team to the next “Edgy Tech Conference.” In order to fill that gap we bring in guest speakers from time to time. We’ve invited people like the creator of Backbone (@jashkenas) and (@mrb_bk), an engineer from Code Climate (which we use for JS/Ruby static analysis).

"What if," you say, "I get a crazy idea that I just want to try and I never have time to build it?" You'll have your chance! In the year I've been here, we've had a hack day, a (3-day) hack “week” and a full, five-day hack week the last week of August. On hack week you get to work on (more or less) whatever you want as long as it’s vaguely related to making RTR more awesome. Best of all, these hack week projects often turn into real, even huge, projects that alter the roadmap of the company! For example, on renttherunway.com we have a feature where users can upload their own photos. Unfortunately, the quality of these photos varies wildly. What if we had a way to automatically give each photo a quality score so that we could show the higher quality photos more prominently? I happen to have a background in machine vision. For my first hack day project, I built a first cut of such an image quality metric. It was simple, but actually showed visible improvements. (In a nutshell, the metric gave preference to images with 1 or 2 faces and images that are overall brighter.) Another example? Our huge new feature, a whole new subscription-based rental model called RTR Unlimited, started life as a hack week project.

But wait! There's more! A recent new hire asked me if RTR_tech does anything with open source. Indeed, we do! For starters, we are committers and some of the main promoters of the aforementioned Dropwizard. We also have more than one OSS project that we produced internally: Alchemy for A/B testing and Conduit for simplifying the use of message queues. I'm certain these projects are just the beginning. In fact, it's worth noting that our internal software development process is modeled to a degree on open source development. We use pull requests for everything, no matter how small, and naturally, have awesome unit test coverage. This is a great way to develop and combines the best of OSS development with the benefits of being in an office right next to your teammates. For one thing, I have found pervasive code reviews to be an excellent way to spin up new people on company standards or languages that are entirely new to them. It really works and has allowed me to remove one worry from my list of "oh man, new person, so much to cover."

All of this sounds fantastic, but we’re a startup so we must be working like dogs, right?

Well...

We have unlimited vacation. Now I admit to being a bit (a lot) cynical. On joining RTR_tech, I assumed that "unlimited vacation" was code for "guilt-based vacation." I was wrong. This is simply not the case. People take lots of time off, myself included, and I haven't seen any guilt-tripping at all. Remember, we're talking about real people here. Awesome people. It turns out, we respect each others’ need for time off. When @MichelleWernick goes to Paris, we all get excited for her and people step up to help fill the gap in QA. When @skamille goes to Hawaii, we don't flood her inbox; we step up and exercise our latent CTO superpowers. When she gets back, there's Hawaiian candy on the kitchen counter and funny stories about emergency room visits.

In fact, it doesn't end with unlimited vacation. We have stellar work/life balance in general. @RTR_tech we understand that the trick is to anticipate, plan, and course-correct a project early. We work hard to avoid feature creep and instead focus on quality. We work smarter, not harder. In this way, we deliver awesome software without people freaking out at the last minute, having to put out fires, and working 12-hour days. Sure, we're not perfect at it, but we've had major successes and we believe in it. We're continuously improving. (How about a Drinks and Demos presentation about what went right and what went wrong in that last big project release? OK!) We embody this in other ways too: Unlimited vacation begets unlimited maternity leave. (I've seen it more than once! It happens! It works! All companies should do it!) And after your maternity leave, maybe you want to have your newborn brought by the office every day so you can take time to feed it. Of course! Who wouldn't allow that? I have a son and a wife. I get to go home and eat dinner with them and when I return to the office the next day, renttherunway.com is still there and I get to do more cool work on it with my fantastic team! I'm more loyal for the stability it provides. I take the time to write a blog entry because of it, and I spend that much more energy recruiting the next awesome teammate because I love telling them about it! What if you don't have a family? What if you're young and unattached? RTR is in NYC! Go enjoy living in New York Effing City!

Did I mention we're encouraged to write articles for our tech blog? Perhaps you've seen said blog? These kind of things are literally listed in our quarterly tech team goals. "1) Ship feature X so we can rent some more dresses. 2) Post 6 tech blog entries."

OK, I promise I'm almost done gushing, but stick with me a bit longer. There's so many more cool things to tell you.

I had a new junior engineer start recently. I gave him a week or two to settle in before I gave him his first major task. Let me set the stage: we're nearing completion of some initial research that uses Python to do Markov Chain Monte Carlo simulation for parameter inference on a Bayesian Network. (To predict the future. NBD.) I asked New Guy to determine if it's viable for us to port that to Java for production use. (The answer, two weeks later, appears to be yes, if a bit inelegantly, via Yadas (Yada Yada Yadas). So that's a thing we do @RTR_tech. Rad, right?

Our interview process is an area we've recently been working to improve (because we're doing lots of it). So how are we doing? Yesterday, I got overwhelmingly positive feedback from a new hire. He thought that it was fantastic to have an RTR_tech engineer guiding him through the whole interview process, answering his questions, being open, honest and enthusiastic. Shortly thereafter, he took the gig. Sounds like it's working!

Did you notice that I used one or two (or forty-two) Twitter handles here? That's because I'd love for you to join our conversation! I want to hear from you! What are we missing? What more can we do? To my fellow RTR_tech-ers (is that a thing?): what is life @RTR_tech to you? Follow us on Twitter @RTR_tech! But don't stop there! Connect with these awesome people about awesome stuff: @skamille, @ericqweinstein, @markwunsch, @bhsdrew, @CarloBarbara, @timx, and many more (use transitive closure to find everyone!)….

This is life @RTR_tech.

P.S. - I really wanted to figure out how I could work in a joke about Talk Like A Pirate Day being a big deal at RTR, but, as any good engineer knows, sometimes you have to kill your little darlings.


Our Stack. Scalability Is Key!

CAB_bio_pic-thumb
CAB_bio_pic-thumb

Intro

We use Java and we like it! When I say this to some people, they cringe. They look at me like I'm a dinosaur who doesn't realize his extinction is coming. I usually smirk, I get it. These are folks that or so in love with Ruby/Rails or Python/Django that they forget it's just a tool for solving a certain kind of problem. I can understand why, they are both great tools! I built my personal site in Rails and it was a lot of fun. It's easy to iterate, and the community makes adding value very simple. The problem is you when you fall in love with your hammer, everything looks like a nail. In this post, I'll tell you some of the problems we faced, and why that made a pure 100% Rails app a bad choice for us. Then we will talk about how we are thinking about leveraging Ruby in our stack. But first, a little history is required.

Our Old Stack

Over three years ago our founders hired some consultants to build an MVP for their vision. They came back with PHP/Drupal and MySQL, a monolithic architecture. It was functional, allowing the founders to start growing the business. Customers were trying it out, and loving the service. The business had legs and traffic was steadily rising. Fast forward to last year, and our infrastructure was handling the flow, but it was creaking loudly. This architecture was fragile, and it didn't scale well in any direction.

Some of the issues we had with the codebase included:

  • Monolithic code base. How do you make changes to one part without releasing everything else? How do you this with 20 devs? 100?
  • How do you improve performance in this architecture? Caching will help some if your data is mostly static. Scaling horizontally at this level may not be not enough to increase throughput, it's too coarse and it requires logical separation to avoid DB concurrency issues.
  • Concerns weren't separated, so one component going down risks taking down the whole site. Ideally you have "swim lanes", basically silos where if any component in the tech stack goes down, only that silo is affected. The product detail page can go down without taking down checkout, the homepage can be unavailable but those on product pages don't notice.
  • How do you support multiple platforms? We don't want to write the same database logic in many different places, and the monolithic code base won't support apps for iphone, android, or ipad.
  • The Drupal Code was spaghetti code. There was SQL in the views and controllers, which meant even making simple view changes was complex and risky. Adding insult to injury, the test coverage was non-existent. This made iterating a dangerous task.

Making it Scale

We like to think of Scalability in terms of the AFK Scalability Cube[1]. There are three axes for scaling you application

  1. The Y - axis (Splitting architecture out by service or function)
  2. The X - axis (Creating N instances of a component, all replicas. Excluding 1 designated for writes. Fronted by a load balancer)
  3. The Z - axis (Splitting resources by user characteristics. i.e The West coast on 1 pool, the East on another)

The further along these axes you get, the better your application scales. We know we needed a highly availability and concurrency, so we needed to scale in multiple directions.

Here is how we made this happen in practice, some of which we are still iterating on:

  • (Y axis) We started dismantling the monolithic beast and migrating to a Service Oriented Architecture (SOA), based on Java backend web services.
    • RESTful web services delivering JSON
    • The concerns are well separated, a service has a specific job and does that job well
    • Easier for development teams to own a full stack of components. This allows us to create small teams which act on goals without spelling out direction, fostering innovation.
    • Allows for more frequent releases
    • Creates disaster isolation: if only that service goes down, the rest of the site should remain available where you don't have cross swim lane dependencies.
    • Why Java? It is easy to test, easier to scale, and more powerful than most high-level languages.
    • (X axis) We have pools of service instances fronted by load balancers.
      • For a service that is slower or more popular, throw more instances in the pool
      • Since the services themselves are smaller than one big app, you can better leverage hardware resources
      • (Y axis) The view is thin, and renders whatever the services provide.
        • Create different view layers for different platforms
        • Services don't care who the client is, they just respond to requests!
        • (All) Metrics gathering. You can't make it better if you can't measure it!

Ruby usage at RTR

When I think about SOA and scalability, I think about the JVM. It's stable, proven, and optimized. When you factor in the size of the community and tools available, it makes a great choice for writing your services. I couldn't see myself writing highly concurrent services in Ruby or Python, but maybe it's because I'm not as familiar with those languages. With that said, I haven't heard of many companies with similar performance requirements going down that route either. Going with Java comes with some overhead. It's verbose, and isn't as convention driven. This means it takes devs longer to write/read code and understand APIs. It also means you need strong leadership pushing solid practices, otherwise you can end up with multiple approaches. This is actually where I think Rails excels, convention over configuration speeds up development. So if you don't require high scalability or have a big dev team, Rails might be the way to go. It's definitely faster time to market, and a better tool for building an MVP.

As we continue to dismantle our monolithic beast and move away from PHP, we want something light and simple that we can control. For that reason, we are moving to Sinatra/Ruby. Stay tuned for more info on how that goes...

-@CarloBarbara

[1] AKF Cube