Recently, I was asked "I was wondering what process would you follow that could build on what we know and dont know from <our hackathon> and your hackathons? We, ... are facing the issues of preparing for recurrence..."
Here was my response
Instead of talking about decisions others ought make, let me talk about decisions I’ve made when faced with a similar problem (albeit over a longer timeframe). My disease has been studied the same way by the same people for decades. OS has not improved in about 14 years. I view this as a “research process” problem (as does the former head of the US FDA).
- Open Participation: Hackathons open investigation to everyone, not just established players. E.g. in 2018, we brought in 17 teams, most of them inexperienced. The bet here is that breakthrough ideas may come from experienced heads, or the may come from “Beginner’s Minds”. You just don’t know.
- Common Deliverable(s): In 2018, teams had a primary common deliverable called “Genes of interest” to enable objective comparison. Note however that although "genes" are important, they might not be the only focus in investigating a treatment.
- Objective Function: Everyone can participate, but we used several scoring mechanisms to figure out who to listen to first. Note that all these scoring mechanism are -objective-, they don’t depend on interpretation by particular persons, and -automated-, I can use computer programs to help scale up the process.
- “Panel of experts”. Here, if the a group (especially an inexperienced one) produced a set of genes that appeared in a corpus of research on the disease, it was ranked higher. The issue with this objective function is that these genes could have just been googled.
- “Results overlap”. If two no-collaborating strangers say the same thing, maybe it makes sense to listen. This issue with this objective function is that agreement does not necessarily mean the genes lead anywhere.
- “Ranking via a ‘tumor-normal RNA-seq differential expression’ holdout set. This highest scoring team and tool using this function were not specialists in my disease. The issue with this objective function is that highly expressed genes are not a guarantee of a tie-in to promising pathway or mechanism.
- Independence: Though the atmosphere is collegial, the teams are independent. If you are aiming at a fly on the wall, and don't know exactly where it is, it's better to use a shotgun than a rifle. In a hackathon, the patient wins if -any- team succeeds. Note that we tracked other factors as well. Of these, "Team Member Count" had the most explanatory power for the objective function scores. That is, small teams had better "objective function" rankings. This tendency has been noted elsewhere.
- Small Budget: These are all low budget affairs, funded by donated facilities, food, research and compute time.
Patient provided Data is the grist for the process. (Patients take note. This is the most important section.)
- Sampling: Although you never know what new analysis techniques will be developed as your disease evolves, it’s still a good bet that any new technique will be informed by samples of patient tissue and blood through time. (e.g. many researchers do longitudinal studies to inform them of disease evolution, among other things.) So patients need to be diligent in gathering their samples.
- Diminishing resource: Fortunately, I had blood and tumor (FFPE and OCT-embedded Tissue) preserved from my 2014 operation. But this is a diminishing resource, so ideally, lots ought be extracted and preserved initially, and deep analysis ought be performed with each piece.
- Genetics: Some researchers believe that -omic analysis of siblings and parents can help. In particular, at a recent Desioid tumor hackathon, the subject's father mentioned for the first time that he had keloids, which pushed the research in a new direction. Also, my sibling was recently diagnosed with thyroid cancer, which has several characteristics in common with my disease. (Thanks to Derya Karaarsian, a Galaxy user, who emphasized this approach. She also provided these citations here and here )