How to choose a research topic
For most of my academic career, my choice of research topic was driven by curiosity, some sort of “gut feeling”, and a desire to make a difference, discover something new and make a contribution towards human knowledge. In practice though, it was also heavily influenced by chance, availability of jobs and the research interests of my supervisors and collaborators.
My perspective has changed since spending time in industry and has been reshaped by the work of effective altruism organisations, in particular 80,000 hours, which aims to help people maximise the good they do with their career.
I have decided to base my criteria for choosing a new research topic on the framework used by 80,000 hours to compare global problems.
I will be taking two sets of criteria into consideration. The first is focussed on the topic itself: the scale of its impact; how many other people are already working on it; and how feasible it is given the time and resources I have. The second is focussed on personal criteria: my skills and expertise; my lifestyle; and my interests.
In order to rank my shortlist of topics, I will use a decision matrix. This is a fantastic tool that I recommend using for any big decision where you have to weigh up multiple factors.
At the end I will share my shortlist and which topic(s) I am going to pursue in the next step of my research process. I will also share a spreadsheet you can use to make your own decision matrix.
Contents
- Topic-centred criteria
- Personal criteria
- Ranking research topics
- My shortlist of research topics
- My top 5
Topic-centred criteria
Prioritise research topics with high impact
Having an impact means having an effect or influence on someone or something. The higher the impact the greater this effect or the larger the number affected.
In data science, the impact (business value) of a project is calculated before it starts and is continuously re-evaluated using a well-defined metric. Companies are pretty ruthless about this. Some exploration is usually acceptable but if a project is not paying for itself, making a profit or drawing in more customers after a pre-determined period of time it will be terminated. This is especially true for machine learning projects where the outcome is often uncertain and the quality of the data is frequently unknown but the potential for future impact is enormous. Having an estimate of its value ahead of time is a requirement.
Measuring the impact of research can be tricky though. Working on something with immediate practical applications may be easier to evaluate but basic research can have far-reaching and unforeseen impacts in the future with no formula or well-defined metric to compute for how it changes the world.
In academia, impact is usually measured by the number of citations and where you publish but this does not necessarily reflect the actual value of the work and it is not known in advance.
It should, however, be possible to get a rough estimate of the value of a research project, by considering the following:
- Who (people/non-human animals/plants/etc.) will be affected?
- How many (local populations/global etc.) will be affected?
- Over what timescale (the next year or the next thousand years)?
- In what way will the impact be felt (reduction of suffering, number of lives saved, quality of life, reduction in CO2 emissions etc.)?
Find a niche
Choosing a topic that is not already saturated with other researchers – a topic that is currently neglected – makes it easier to make a significant contribution and less likely that someone else would do the work if you didn’t.
Working on a topic that not many others are working on will give you a niche and allow you to focus your learning, networking and skill development to become an expert.
80,000 hours has a lot more on neglectedness.
Work on something feasible
It is tempting to work on a huge problem that no one has solved for decades but think carefully before jumping in. I spent my PhD working on the protein folding problem and it was hugely frustrating at times. With the help of my supervisors, I learnt how to limit the scope to something much more feasible that still made a contribution.
How feasible a research topic is will depend on how much time and resources you have. Ideally, you want to work on something with an ambitious goal that also gives you milestones along the way that each have a positive impact in their own right.
To start with, I indend to choose topics where I can make a contribution within a year using only my laptop (plus cloud computing).
Personal criteria
Make use of your skills and expertise
Use your skills and existing expertise to narrow down what topic you choose. List all the skills/expertise you have and the ones you can, or want to, master in a reasonable time. Rank these by how competant you are, or expect to be, in each. Then focus your search on topics that make the most of the top three.
This will mean you make progress quickly and will likely find the work more satisfying on a day-to-day basis.
My top three skills/expertise are:
- Machine learning
- Programming
- Biology
I will try to find a topic that uses all three of these and I will exclude any topics that use none of them.
Fit your work to your lifestyle
For me, lifestyle has become a more important factor as I’ve got older. Earlier in my career I was happy to move around and live in different countries. This was also necessary since staying in one place is actively discouraged in academia. I also didn’t mind travelling to conferences and working in the lab at weekends or (less enjoyable) preparing seminars during the holidays. But this kind of lifestyle can become more difficult, and less desirable, when you have a partner, children (and pets!)
You may also have hobbies that are important for your well-being and require time or have their own geographical constraints.
You will be less productive if you are unhappy with how your work fits into other areas of your life, so I recommend taking lifestyle into account. There is a big difference between working on something that can only be done at night in a molecular biology lab and working on something that can be done on a computer at home.
I will be working on topics that can fit around my consulting work, the school run and the schedule of our new puppy!
Work on something that sparks your curiosity
Finally, curiosity and wonder are key for maintaining focus and perseverance, both of which are required to succeed in research. Things often don’t work as well as you’d hoped and you will probably find yourself at a deadend on multiple occasions. It is important that you are interested enough in your topic to overcome obstacles, start again, spend time learning something new and push through the more tedious tasks (of which there will be many).
Interest can also develop from work that you enjoy doing and that gives you purpose, if you don’t already have a burning desire to work on a particular topic.
I knew from an early age that I wanted to be a scientist. The only trouble I had getting there was choosing between biology and astrophysics. My dream job, I decided, was finding life on other planets and/or figuring out what planets life could live on. This feeling has stayed with me throughout my career but I never found a path to get there. Until now! So I will be including some astrobiology topics in my list of candidates.
Ranking research topics
I am using a weighted decision matrix to prioritise what I work on. A decision matrix is a fantastic tool for ranking different options (rows) according to multiple criteria (columns). It is also a great learning process trying to determine the scores - you find out where the gaps in your knowledge are about a particular option. I used it when my husband and I were trying to decide where to live for the next 10 years. It led us from Norway to the U.K. I wish I had discovered it earlier - it would have saved me making many a list of pros and cons.
A decision matrix looks like this:
| Options | Criterion #1 | Criterion #2 | Criterion #3 | …. | Criterion #n | Score |
|---|---|---|---|---|---|---|
| Weight –> | w1 | w2 | w3 | … | wn | |
| Option #1 | … | |||||
| Option #2 | … | |||||
| Option #3 | … | |||||
| … | … | |||||
| Option #m | … |
How to use a decision matrix
- List all your options in the options column (in any order)
- List all your criteria as additional columns (in any order)
- Decide on a weight for each criterion, according to how important it is relative to the others
- If the criteria are all of equal importance then the weights are all 1
- If one criterion is twice as important as the others it will have a weight of 2
-
For each criterion, go through each option and give it a score. Depending on the criterion, you may have an accurate number or you may need to guestimate. Some of the scores will be highly subjective and dependant on your world view.
In any case, to keep the formula for computing the overall score simple, it is important to use the same scale for all criteria (I like using a scale from 0-10) and for higher to always mean better.
You can convert any set of scores to any other scale by subtracting the minimum value of the original range of values, dividing by the original range then multiplying by the new range and adding the minimum value of the new range (e.g. 91% would become 9.1 on a scale from 0-10):
\[x' = {a} + {(x - min(x))(b - a) \over max(x) - min(x)}\] - Compute the final score for each option by multiplying each criterion score by its weight and adding them all together (using the
SUMPRODUCTfunction in a spreadsheet) - Sort all the options by their overall score.
- Consider adding some filters if you want to ensure certain criteria are met at a certain threshold
- Your best option according to your scoring is now at the top!
Where to start
A decision matrix is a great tool for deciding between different options, but first you need to figure out what your options are (and also what criteria are important to you).
Get an overview of your options
There are several organisations that have published lists of priority areas or open research questions online. These are a good place to start if you want to maximise impact:
- 80,000 hours list of research project ideas
- Effective Altruism Forum directory of open research questions
You can also get some inspiration by browsing Nature, Science, Cell, or the top journal in your field. These are the articles that editors and reviewers decided were high impact.
To get an overview of a more unfamiliar field (or to give yourself a refresher), look at the contents pages of some undergraduate textbooks.
If you have multiple interests, try combining them to give additional ideas.
Try brainstorming or mind-mapping.
No idea is a bad idea at this stage – let the decision matrix filter these out for you.
Decide what criteria are important to you and how you will measure them
I broke impact, neglectedness and feasibility down into the following columns:
- Impact
- Number of people affected (scale 0-10)
- Number of non-human animals affected (scale 0-10)
- Increase in well-being/health/reduction in suffering (scale 0-10)
- Neglectedness
- Lack of existing researchers (scale 0-10)
- Lack of existing funding (scale 0-10)
- Feasibility
- Availability of existing resources (scale 0-10)
- 10 - years to complete work
For the personal criteria, I used the following:
- Fit to lifestyle (scale 0-10)
- Use of AI skills (scale 0-10) (I gave this a higher weighting than my other skills)
- Use of programming skills (scale 0-10)
- Use of biology expertise (scale 0-10)
- Interest (scale 0-10)
Download my decision matrix template
I have created a template that you can download and fill in with your own data using Excel or Google sheets:
You can see my version below.
My shortlist of research topics
The table below is the end result of my decision matrix (here is the spreadsheet version).
| Impact | Neglectedness | Feasibility | Personal criteria | Score | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Topic | People affected (scale 0-10) | Non-human animals affected (scale 0-10) | Increase in well-being/health/reduction in suffering (scale 0-10) | Lack of existing researchers (scale 0-10) | Lack of existing funding (scale 1-10) | Availability of existing resources (scale 0-10) | 10 - years to complete work | Fit to lifestyle (scale 0-10) | Use of AI skills (scale 0-10) | Use of programming skills (scale 0-10) | Use of biology expertise (scale 0-10) | Interest (scale 0-10) | ||
| Rank | Weight | 2 | 2 | 2 | 1 | 1 | 1 | 1 | 2 | 2 | 0.5 | 0.5 | 2 | |
| 1 | AI for translating unseen languages | 5 | 6 | 8 | 7 | 9 | 9 | 9 | 10 | 10 | 10 | 8 | 9 | 139 |
| 2 | Extraterrestrial technosignatures | 8 | 3 | 7 | 9 | 9 | 8 | 9 | 10 | 10 | 10 | 3 | 10 | 137.5 |
| 3 | Recognising AI sentience | 8 | 4 | 7 | 9 | 9 | 9 | 8 | 10 | 10 | 10 | 5 | 8 | 136.5 |
| 4 | Biosignatures | 8 | 4 | 8 | 5 | 5 | 8 | 8 | 10 | 10 | 10 | 9 | 10 | 135.5 |
| 5 | AI misuse: pathogenic DNA | 9 | 5 | 8 | 7 | 6 | 9 | 9 | 10 | 10 | 10 | 8 | 5 | 134 |
| 6 | Encoding human values into AI | 8 | 3 | 8 | 9 | 9 | 9 | 8 | 10 | 10 | 10 | 2 | 5 | 129 |
| 7 | Measuring sentience | 8 | 8 | 8 | 9 | 9 | 9 | 7 | 10 | 2 | 5 | 7 | 8 | 128 |
| 8 | Monitoring AI values | 8 | 4 | 7 | 9 | 9 | 9 | 7 | 10 | 10 | 10 | 1 | 5 | 127.5 |
| 9 | Existential risks of deforestation and ocean pollution | 9 | 8 | 8 | 5 | 5 | 7 | 8 | 10 | 8 | 5 | 5 | 5 | 126 |
| 10 | Interstellar travel | 9 | 4 | 9 | 9 | 9 | 7 | 9 | 10 | 5 | 1 | 1 | 8 | 125 |
| 11 | Measuring suffering of non-human animals | 4 | 8 | 9 | 9 | 9 | 6 | 7 | 10 | 5 | 5 | 7 | 8 | 125 |
| 12 | Habitable planets | 5 | 3 | 6 | 4 | 5 | 7 | 8 | 10 | 10 | 10 | 7 | 10 | 120.5 |
| 13 | AI misuse: adversarial attacks on critical AI systems | 9 | 3 | 7 | 5 | 5 | 9 | 8 | 10 | 10 | 10 | 1 | 5 | 120.5 |
| 14 | Protecting biodiversity in the far future | 7 | 8 | 8 | 9 | 9 | 7 | 7 | 10 | 1 | 1 | 7 | 8 | 120 |
| 15 | Evolution and biological mechanisms of stress responses across the tree of life | 4 | 8 | 6 | 6 | 5 | 7 | 9 | 10 | 1 | 1 | 10 | 9 | 108.5 |
| 16 | Spatiotemporal landscape of human values | 5 | 6 | 4 | 7 | 7 | 9 | 9 | 10 | 1 | 1 | 2 | 8 | 101.5 |
| 17 | Emergent behaviour of human populations and existential risk | 5 | 3 | 4 | 7 | 5 | 9 | 7 | 10 | 5 | 10 | 1 | 5 | 97.5 |
My top 5
Since I am somewhat uncertain about my scoring (there was a lot of guessing), my next step is to spend a day learning more about each of my top 5 topics before deciding which of these to make a start on:
- AI for translating unseen languages
- Extraterrestrial technosignatures
- Recognising AI sentience
- Biosignatures
- AI misuse: pathogenic DNA
I am excited about all of these!
Share your thoughts
How do you decide what to work on? Do you have a favourite tool for making decisions? Do you have a better way of estimating impact? Any suggestions for research topics? Let me know in the comments below.
Comments