Erika Update #11: Finally a breakthrough! If you can’t beat it, measure it

Sanity is restored

Much has been written about “eureka moments” in science. I had a miniature breakthrough of my own about eight months into my main grad school project.

The project faced a real blocker. Many of the biomolecules I wanted to engineer were too toxic for me to measure reliably. Overcoming this obstacle involved digging deeper than I thought I was necessary or appropriate. As I say in Erika Update #11, linked below, “I think there’s a lot more that can be learned from the kinetic data, which will feel silly and pedantic to study, but should prove useful.” 

Investing time in fully understanding what was happening paid off. I finally created a reliable way to gather data that wasn’t fooled by toxic biomolecules! It was a real turning point in the project. I leveled up.

Erika Update #11

Here’s Erika Update #11 - 2018 8 8 - kinetic luminescence assays

Recall that going into this experiment I knew that there was something fishy going on with my measurement technique. In Erika Update #8 and 9, I had isolated an example of a qtRNA I couldn’t reliably measure. I knew that the problem seemed to be that toxic qtRNAs would sometimes ‘fake out’ my assay and look like they were functional. 

My approach to developing a good assay was to add ALL the controls (functional and non-toxic, functional and toxic, non-functional and non-toxic, non-functional and toxic) and confirm that I could correctly classify the functionality and toxicity of every sample from the data as if I didn’t know how they should turn out. I have a breakdown of all the controls I added on page 2, and as I put it, “we win when we’re not faked out by” the signal from my toxic-but-ground-truth-is-that-it’s-actually-not-functional negative control.

And indeed, lo and behold, if you measure these qtRNAs as they grow, it reveals much richer data that allows you to distinguish toxic-but-not-functional from toxic-and-functional qtRNAs! In the update, you can see this in the “GlnU Lum(OD) graph on page 6: the bright blue and dark blue lines (functional but toxic) are distinguishable from the bright orange and dark orange lines (non-functional but toxic). Win!!

Note the date on this update. Whereas before I had been writing an update every ~2 weeks, it took me more than a month to wrap my head around what might be happening, design controls, execute the experiment, and analyze all the results. It was well worth it.

Where’d it end up?

Measuring qtRNAs using this assay was the reliable measurement technique that I used moving forward. However, somehow, despite preparing a really nice supplemental figure that describes this measurement technique, it never made it into any of the three papers that use this data.

Actual snapchat footage of my desk at the Broad. I had finally figured out that using a label printer enabled me to do experiments as big as any spreadsheet I could create on my computer - all without making any typos!

And to orient you

I think pretty much everyone can get something out of reading these blog posts + updates. Here’s some additional notes customized for you, depending on your interests + background:

• You’re a non-scientist: Science requires keeping careful track of what you know, and what you don’t know. Measuring something new about something that has never been measured before is hard! Before you do anything else, you need to convince yourself you can measure anything at all, usually with existing known test cases like I did here. Only then can you move on to measuring actually new things.

• You’re a student doing a PhD/undergrad research/etc: On the top of page 4 I mention that I experimented with trying to fit a sigmoid to my growth curve data, and it wasn’t reliable enough for me to trust it as the quantification technique. I now know of several packages for analyzing plate reader data, like this great Nature Protocols paper.

• You have ideas for reforming publishing: Part of the challenge with disseminating nitty-gritty gory details for how to make science work is that nobody is ever going to read it. When I sent out my Erika-Update to the email thread, even my mentor Ahmed didn’t read it properly! I’m certainly not the first person who has figured out that you need to take kinetic measurements in order to handle toxic biomolecules. The problem is that many things prevent us from sharing this sort of information with other scientists. 1) It’s boring. 2) Explaining why this matters is tricky and subtle. 3) The controls you use are system-specific, so even if someone else had written about this it still requires mental gymnastics to apply it to your own science. 4) I don’t know how to search for this sort of knowledge in the literature. What would I google? “Help, my plate reader experiment doesn’t make sense”?

Want more?

If you want to follow along with this project, you can get updates by signing up through substack, or following me on linkedin or twitter.

If you have ideas for what I should cover in the blog post, suggestions for vocabulary to define, questions about the science, or other comments, please do reach out by twitter DM - I’d love to hear from you!