Recently, I was asked "I was wondering what process would you follow that could build on what we know and dont know from <a current hackathon> and your hackathons? We, ... are facing the issues of preparing for recurrence..."
Here was my response
Instead of talking about decisions others ought make, let me talk about decisions I’ve made when faced with a similar problem (albeit over a longer timeframe). My disease has been studied the same way by the same people for decades. OS has not improved in about 14 years. I view this as a “research process” problem (as does the former head of the US FDA).
So, the question is then, what are the key elements of the proposed alternative?
- Open Participation: At my hackathons, I open investigation to everyone, not just established players. E.g. in 2018, I brought in 17 teams, most of them were inexperienced. The bet here is that breakthrough ideas may come from experienced heads, or the may come from “Beginner’s Minds”. You just don’t know.
- Common Deliverable(s): In 2018, teams had a primary common deliverable called “Genes of interest” to enable objective comparison. The issue with this is that genes may not be the best way to investigate this malady. Note that I tracked other factors as well. Of these, "Team Member Count" had the most explanatory power. That is, small teams had better "objective function" rankings (next item).
- Objective Function: Everyone can participate, but I used several scoring mechanisms to figure out who to listen to. Note that all these scoring mechanism are -objective-, they don’t depend on interpretation by particular persons, and -automated-, I can use computer programs to help scale up the process.
- “Panel of experts”. Here, if the a group (especially an inexperienced one) produced a set of genes that appeared in a corpus of research on the disease, it was ranked higher. The issue with this objective function is that these genes could have just been googled.
- “Results overlap”. If two no-collaborating strangers say the same thing, maybe it makes sense to listen. This issue with this objective function is that agreement does not necessarily mean the genes are correct.
- “Ranking via a ‘tumor-normal RNA-seq differential expression’ holdout set. This highest scoring team and tool using this function were not specialists in my disease. The issue with this objective function is that highly expressed genes are not a guarantee of a tie-in to pathway or mechanism.
Data is the grist for the process. (This is the most important section.)
- Sampling: An article of faith. Although, you never know what new analysis techniques will be developed as your disease evolves, it’s still a good bet that any new technique will be informed by samples of patient tissue and blood through time. (e.g. many researchers do longitudinal studies to inform them of disease evolution, among other things.) So gather the samples.
- Diminishing resource: Fortunately, I had blood and tumor (FFPE and OCT-embedded Tissue) preserved from my 2014 operation. This is a diminishing resource, so ideally, lots ought be extracted and preserved initially, and deep analysis ought be performed with each piece.
- Genetics: Some researchers believe that -omic analysis of siblings and parents can help. In particular, it may not show what the problem is, but can help determine what the problem isn’t.