- This Week

- News in Focus

- Books & Arts

- Opinion

- Work

- Research

This Week

- End the glaring inequity in international science collaborations

- From Einstein to AI: how 100 years have shaped science

- The postdoc experience is broken. Funders such as the NIH must help to reimagine it

- Research prizes are opaque and rife with bias — it’s time to shake them up

EDITORIAL 20 December 2023

End the glaring inequity in international science collaborations

The world’s natural-science research ecosystem remains focused on the priorities of high-income countries. Funders, publishers and scholarly databases can do more to help to rebalance that.

Researchers at the University of Ghana find themselves caught between needing to publish in international journals and the necessity of supporting African publications.Credit: Gabriela Barnuevo/AP Photo/Alamy

Over the past decade, the importance of improving equity in global research collaborations has gained increased attention worldwide. However, data published last week by the Nature Index in its first-ever supplement on global north–south research collaborations show just how much work still needs to be done to bridge a yawning gap (see go.nature.com/3rs5xdm).

Across the 82 natural-science journals tracked by the index, just 2.7% of articles published between 2015 and 2022 featured collaborations between scientists in higher-income and lower-income countries. Even in these articles, there were, on average, three authors in richer countries for every one author in a poorer country. And the number of articles that involved collaborations between researchers exclusively in poorer nations was just 24 — out of a total of some half a million articles.

The Nature Index supplement divides countries into two categories on the basis of four income groups used by the World Bank: the global north, which includes high-income and upper-middle-income countries, and the global south, comprising lower-middle-income and low-income nations.

In articles featuring north–south collaboration, almost half of the author contributions came from just five wealthy countries — China, France, Germany, the United Kingdom and the United States. Among global-south countries, India’s contribution represented 15% of all north–south research. By contrast, 42 African countries had a collective contribution amounting to less than 20% of that of India.

For many, it will come as no surprise that richer countries dominate collaborations. But the extent of this inequity must be a wake-up call for funders and publishers, which should not allow the status quo to continue.

The Nature Index tracks publications and authorships, focusing on a select tranche of journals in which, according to its own data, global-south researchers struggle to publish. It needs to recognize that quality research from the global south might not be reaching this subset of publications, and take steps to address the imbalance. The index is already in the process of broadening the scope of the subjects it covers, and as part of this will consider the publications and other venues through which global-south researchers share their work.

Notably, until earlier this year, the Nature Index focused on only the natural sciences, and its data reflect that. Separate data from the Digital Science Dimensions database show relatively more north–south collaboration in the health sciences and engineering, which the index team hopes to assess in future.

This caveat aside, the findings reported in the Nature Index supplement represent the reality for many scientists in lower-income countries. In Who Counts, an open-access study published as a book at the start of this year, social scientist David Mills and his co-authors relate the experiences of researchers at two universities in Ghana trying to navigate the international science-publishing system. They point out the pressure researchers feel under to publish internationally to develop their careers, and describe the higher costs of this, in terms of time and resources. The alternative is to support local journals and publishing, but often African journals are not recognized by international scholarly databases. The authors call this “bibliometric coloniality”: higher-income countries setting the rules for what does and doesn’t ‘count’ as measured research outputs.

This account chimes with the thinking behind the Africa Charter for Transformative Research Collaborations. This was launched in July by a coalition of organizations with interests in higher education and research on the continent. Signatories, which include the African Academy of Sciences and the Association of African Universities, want publishers to do more to recognize research and collaboration by the continent’s scientists, and to allow countries in the global south to drive their own knowledge creation, rather than become trapped in an agenda set by richer nations.

There are also ways in which the existing data could be examined and presented to give research institutions and funders the tools they need to target and reward research that shifts the north–south imbalance. It is crucial that analyses distinguish between major global-south hubs that already have reputation and funding advantages and lesser-known institutions and countries where capacity needs building. And publishers handling papers resulting from north–south collaborations should ensure that authorship conventions don’t stop researchers in poorer nations getting sufficient credit for their work.

Research and collaboration are widespread across low- and lower-middle-income countries, but, all too often, one of the greatest struggles for those involved is gaining international recognition for their work. Funders must give more priority to projects conceived and led by researchers in the global south, and must provide more training partnerships that help to build, rather than drain, resources in lower-income countries. And more journals must enact policies to prevent ‘helicopter’ or ‘parachute’ research, whereby scientists from rich countries visit poorer ones and exploit local expertise or resources.

Databases such as the Nature Index that report on north–south collaborations must find ways to measure progress in the representation of researchers in the global south, alongside a broadened subject and publication scope. This could provide a ‘feedback loop’ for governments and funders to reward those projects helping to move the dial in favour of global-south science.

Nature 624, 473 (2023)

doi: https://doi.org/10.1038/d41586-023-04022-1

Reprints and permissions

EDITORIAL 19 December 2023

From Einstein to AI: how 100 years have shaped science

Looking back a century reveals how much the research landscape has changed — and how unclear the consequences of scientific innovation can be.

A century after the invention of bakelite (widely used to make early phones), talks are under way to agree on a treaty to end plastics pollution.Credit: David Bagnall/Alamy

Earlier this year, Nature published a paper that concluded that science is getting less disruptive[1]. Looking back a century might seem to support that idea. The twentieth century began with a revolution in physics. In 1900, Max Planck laid the foundation for quantum theory. This was followed by Albert Einstein’s annus mirabilis: in 1905, he published four groundbreaking papers on the photoelectric effect[2], Brownian motion[3], the special theory of relativity[4] and the mass–energy relationship[5] described by his famous formula, E = mc^2. Subsequent decades saw the establishment of the general theory of relativity and that of the field of quantum mechanics.

Other scientific areas also saw rapid developments. In 1910, US geneticist Thomas Hunt Morgan used the fruit fly Drosophila to show how genes reside on chromosomes — a crucial step on the path to modern genetics. That same year, Marie Curie successfully isolated pure radium (element 88 in the periodic table). And, in 1925, Australian anthropologist Raymond Dart’s description of an Australopithecus africanus skull provided the first evidence that Africa is the cradle of humankind[6].

Other scientific breakthroughs would shape people’s lives in more practical ways. In 1907, Belgian chemist Leo Baekeland commercialized an invention that he called bakelite — the forerunner of today’s plastics. The material was made up of long, unbreakable chains of hydrocarbon molecules. It didn’t conduct electricity, was mouldable, heat resistant and rather easy on the eye when dyed.

And in 1909, German chemist Fritz Haber discovered a method for producing ammonia, which he and fellow chemist Carl Bosch commercialized at the German chemical company BASF in 1913. Their process of manufacturing ammonia by fixing nitrogen from the air became the basis of the fertilizers that remain crucial to global food security today.

The scientific landscape has changed so much that it would be unrecognizable to someone who lived 100 years ago. The scale of science and innovation, performed by large, globally collaborating teams, and how it is funded (predominantly by industries) would be utterly alien to scientists of old. How research is disseminated to scientific peers and to society would be both foreign and familiar; papers are still published but that is only part of how science is now communicated. And researchers bear many new ethical, legal and societal responsibilities.

It’s hard to argue that some of the discoveries of the twenty-first century so far haven’t been disruptive, in the sense of providing new directions for science. Through global collaborations and with help of multinational funding, scientists produced the first draft sequence of the whole human genome[7] in 2001 and found a way[8] to edit genes efficiently in 2012. These achievements also enabled researchers to swiftly develop mRNA vaccines during the COVID-19 pandemic.

Fundamental physicists discovered the Higgs boson[9],[10] in 2012, nearly 50 years after its prediction. And in 2015, gravitational waves were first detected directly[11], almost 100 years to the day after general relativity provided a theoretical basis for their existence.

Science and society have changed in other ways, too. The past century has taught researchers a lot about the risks of innovations such as plastics and artificial fertilizers. In response, countries have established legally binding agreements through the United Nations to limit the harms of scientific and technological innovations.

Baekeland’s life-changing plastics are now the subject of talks to limit their pollution. The process for producing ammonia is controlled by at least two international conventions. The first intends to limit, or reduce the risks of, greenhouse-gas emissions from production of this chemical. The second is a treaty to eliminate chemical weapons, an application of Haber’s invention that he supported during the First World War.

Recent developments, such as artificial intelligence (AI) technologies, are yet to be governed by global agreements, but they must be as well. Large language models and generative AI — this year’s biggest disruptive innovations — need to be applied such that their potential to do harm does not outweigh their benefits. Nature regularly reports on the challenges posed by generative AI technologies and the current lack of regulation. At some point, such systems will need to be regulated by globally coordinated agreements, as is the case for innovations such as nuclear materials, drugs and vaccines.

It is impossible to predict precisely what impacts this century’s innovations will have 100 years from now. But it is safe to say that the world’s societies, economies and environment will once again have changed, possibly beyond recognition. All the more reason for the international community to continue coordinating regulatory responses to new inventions, such as AI technologies — to avoid disruptive innovations that do more harm than good.

Nature 624, 474 (2023)

doi: https://doi.org/10.1038/d41586-023-04021-2

References

1. Park, M., Leahey, E. & Funk, R. J. Nature 613, 138–144 (2023).

2. Einstein, A. Ann. Phys. 17, 132–148 (1905).

3. Einstein, A. Ann. Phys. 17, 549–560 (1905).

4. Einstein, A. Ann. Phys. 17, 891–921 (1905).

5. Einstein, A. Ann. Phys. 18, 639–641 (1905).

6. Dart, R. A. Nature 115, 195–199 (1925).

7. International Human Genome Sequencing Consortium. Nature 409, 860–921 (2001).

8. Jinek, M. et al. Science 337, 816–821 (2012).

9. ATLAS Collaboration. Phys. Lett. B 716, 1–29 (2012).

10. CMS Collaboration. Phys. Lett. B 716, 30–61 (2012).

11. Abbott, B. P. et al. Phys. Rev. Lett. 116, 061102 (2016).

Reprints and permissions

WORLD VIEW 19 December 2023

The postdoc experience is broken. Funders such as the NIH must help to reimagine it

Postdoctoral woes, especially in biomedical research, are symptomatic of a wider problem. Funding bodies have an opportunity to spearhead a cultural shift.

By

Ubadah Sabbagh

It’s not breaking news that postdocs feel undervalued and demoralized. In a Nature survey of 3,838 postdocs in 93 countries earlier this year, only 40% were satisfied with their salaries, and just 34% with their career-advancement prospects (see go.nature.com/48msp). In the life sciences, early-career researchers are leaving academia in droves.

I know first hand how real the problems are, from my experience both as a postdoc and in my position as the chair of the postdoc and student advisory committee of the Society for Neuroscience in Washington DC. I am one of two postdocs in a working group that was set up by the US National Institutes of Health (NIH) in November 2022 to examine these issues. We received about 3,300 comments between February and April, and gave a preliminary update on our deliberations in June.

Last week, we released a report with six major recommendations on how to reshape the postdoctoral experience (see go.nature.com/3tj2qw), starting with ensuring that all NIH-supported postdocs receive employee benefits and a salary that is commensurate with their skills, adjusted for inflation. Rather than repeating our report, I would like to provide my thoughts on its broader context.

The recommendations are not insignificant. They are a fundamental shift in how the scientific ecosystem perceives and values the role of postdoctoral scholars. The NIH director will now consider them.

I hope that the NIH will not just make the suggested policy changes, but that it will lead the way in transforming the entire academic biomedical research enterprise in the United States. By embracing bold changes and overcoming historical inertia, we ensure fairer compensation for postdocs, clearer career paths and an environment in which scholars from marginalized groups are empowered and foreign talent is invited and retained.

Fair compensation is the most urgent issue — 90% of the comments that the working group received were about pay and benefits. In Boston, Massachusetts, where I live, many postdocs’ salaries are low enough that they are eligible for low-income housing lotteries. The NIH National Research Service Award, a common postdoctoral fellowship in the life sciences, includes a childcare allowance of only US$2,500 a year. The average yearly cost of childcare in Boston for one infant is about $20,000.

But treating postdocs right is not just about raising salaries and improving benefits. The postdoc crisis is not a stand-alone problem. The scientific system that has served us well since the Second World War is no longer viable, and failing to meaningfully change it threatens not just postdocs, but research as a whole. Postdocs are arguably the node in the system most under pressure right now, but that pressure extends to others, too: junior and senior faculty members, graduate students and more.

This broader challenge of reimagining how science is done demands the attention of all institutions. The NIH does not have the authority to ensure that all postdoctoral scholars are appointed as full-time employees at their institutions. But the NIH — and similar funding bodies in other nations and other fields — has a leadership. It can articulate expectations and push forward by engaging with stakeholders, funders, foundations and the institutions that receive its grants.

This includes engaging with US government agencies such as the Internal Revenue Service, Citizenship and Immigration Services and the Department of Homeland Security. A coherent, comprehensive federal strategy could better the lives of US postdocs by improving visa processes and boosting incentives to attract foreign talent (as President Joe Biden’s executive order on artificial intelligence, issued in October, does). Clarifying the tax policy around fellowships and creating new incentives are essential. Every year, graduate students and postdocs with federal fellowships struggle to fully understand how to file their taxes, because the tax code can be confusing.

We also need to redefine both what success looks like and which career paths postdocs are trained for. Becoming a principal investigator at a research university cannot be the definition of success, with everything else being labelled as an ‘exit’ or ‘alternative’. Most data suggest that only about 20% of postdocs will land in tenured or long-term academic positions. Other posts should be considered successful outcomes too, including those at minority-serving institutions, liberal-arts colleges, community colleges, non-profit research institutions, policy think-tanks, biotechnology and pharmaceutical companies and other types of industry. The NIH’s leadership can shift cultural thinking about this topic. Revising what career outcomes are considered successful after completing a NIH-funded training programme would be a simple start. If these necessary changes feel radical or impossible, we need to think about what kind of institutional inertia is working against them, and why.

Any efforts to make large changes should include the voices of early-career scientists. We will be the ones to push science forward. Any effort without us will not be fully serious, meaningful or substantive.

How the scientific system treats researchers is a choice. It is not inevitable or out of our control. It’s time we make bold choices as a scientific community that align with both reality and our values.

Nature 624, 475 (2023)

doi: https://doi.org/10.1038/d41586-023-04028-9

Reprints and permissions

Competing Interests

U.S. served on the NIH director’s advisory committee working group on postdocs from December 2022 to December 2023.

WORLD VIEW 19 December 2023

Research prizes are opaque and rife with bias — it’s time to shake them up

Awards should be standard-bearers for transparent, robust research that is inclusive, equitable and trustworthy. But that is not what they are right now.

By

Malgorzata Lagisz

Almost every academic newsletter I receive nowadays includes announcements of somebody winning an award. But below their shiny surface, I find that such announcements carry a whiff of ambiguity. I start thinking about how winners are selected, who is left out and why, and whether the research community could do this award-giving business better.

Research that I and my colleagues have done shows that things could indeed be done a whole lot better. Scientific prizes are plagued by opaque and seemingly biased selection criteria. This needs to change. Done right, such awards could provide an opportunity to recognize and value transparent and robust research, and build a more inclusive and trustworthy way of doing science.

Big awards, such as Nobel prizes, are often surrounded by public controversy because of a lack of diversity among the winners, and because the selection processes and policies are inequitable or lack transparency. We focused instead on the innumerable smaller awards, administered mainly by journals and learned societies, with categories such as ‘best paper’ or ‘most promising young researcher’. These don’t get as much attention, but they are often stepping stones for career advancement, especially for early- and mid-career researchers. Such awards can filter and reinforce what is considered excellent research.

My team and I — a diverse group of volunteers representing six continents and many career stages — set out to gather data about the transparency and declared values of these smaller, research-focused awards.

We started with an international selection of 13 ‘best researcher’ and 10 ‘best paper’ awards in my area of research, ecology and evolution. The results were published this year (M. Lagisz et al. Nature Ecol. Evol. 7, 655–665; 2023). A larger team has since expanded the assessment to a broad sample of 222 best-paper awards across all disciplines, the results of which were posted as a preprint on 12 December (M. Lagisz et al. Preprint at bioRxiv https://doi.org/k8rr; 2023).

We found that descriptions of the selection criteria and processes used for these awards are generally short and vague. Often, no contact information is given should you wish to request more information. Around half the awards surveyed in our latest study had journal editors involved in nominating or selecting the winners, but 91% did not state how potential conflicts of interest would be handled.

Furthermore, award descriptions rarely mention concepts that align with open science — the movement to make science accessible to all. The only positive example included ‘transparency of the methods’ in its evaluation criteria.

Of the 222 awards, 21 mentioned considering impact metrics — counts of citations or downloads — in their selection process. Concerningly, eight used such measures as the only metric for selecting the ‘best’ paper (there is a separate class of ‘impact’ awards, but we did not include those in our analysis). And although many scientific organizations and institutions claim publicly to be committed to equity, diversity and inclusivity, only two of the 222 awards mentioned related values or policies in their award description and selection processes.

The lack of explicit standards for evaluating science allows assessors to vary their scores depending on the identity of the nominees. Such biases can be compounded when potential or actual conflicts of interest exist and are not managed. Awards that rely on simplistic metrics, such as citations, contribute to an academic ‘Matthew effect’ — ‘to those that have, more shall be given’. As with other indicators of scientific esteem, including numbers of articles published and grants obtained, citations are easier to achieve by some scientists, helping them to secure promotions, jobs and further funding, snowballing into more and bigger awards.

Our data show that between 2001 and 2022, 61% of individual winners were men. Although that finding might align with broader employment patterns in research, we found no discernible trend towards a greater representation of women. Some 48% of winners were affiliated with US institutions. Researchers based in low- and middle-income countries made up just 11% of winners, with more than half of these based in China. This imbalance was particularly marked in the earliest part of the study period, from 2001 to 2010.

Omnipresent prizes and awards reflect scientific communities’ values ‘in action’. We have concluded that they are currently failing to match global calls for improving transparency and equitability in science. Changing how they operate and what they reward can incentivize better research practices and support the drive to open science. Given the slow progress in addressing the many biases prevalent in academia, historically under-represented and marginalized groups can benefit from award-giving institutions, reducing ambiguity and explicitly fostering equitable access and assessment practices.

So, next time you see another award announcement, maybe reflect on whether this prize contributes to the reproducibility crisis and various biases rampant in academia. Is it transparent and equitable? Does it recognize robust and reproducible science? And if you are one of the many people who manage existing awards or are working to establish new ones, now is the time to act and embrace the principles and values of a more inclusive science.

Nature 624, 476 (2023)

doi: https://doi.org/10.1038/d41586-023-04027-w

Reprints and permissions

Competing Interests

M.L. has unpaid membership and governance roles in the following non-profit organizations: the Society for Open Reliable and Transparent Ecology and Evolutionary Biology; the Association for Interdisciplinary Meta-research and Open Science; the European Society for Evolutionary Biology; and Society for Research Synthesis Methodology.

News in Focus

- More than 10,000 research papers were retracted in 2023 — a new record

- ‘Biocomputer’ combines lab-grown brain tissue with electronic hardware

- How a bullying scandal closed a historic astronomy department

- ‘It’s all gone’: CAR-T therapy forces autoimmune diseases into remission

- COP28 climate summit signals the end of fossil fuels — but is it enough?

- RNA biologist loses disability case against Howard Hughes Medical Institute

- The best science images of 2023 — Nature’s picks

- Nature’s 10: ten people (and one non-human) who helped shape science in 2023

- The engineer who helped India to reach the Moon

- Amazon protector: the Brazilian politician who turned the tide on deforestation

- Making mice with two dads: this biologist rewrote the rules on sexual reproduction

- Nuclear-fusion breakthrough: this physicist helped to achieve the first-ever energy gain

- How Earth’s first global heat officer is tackling climate change

- OpenAI’s chief scientist helped to create ChatGPT — while worrying about AI safety

- Superconductivity debunker: this physicist exposed flaws in a blockbuster claim

- Weight-loss-drug pioneer: this biochemist finally gained recognition for her work

- Malaria fighter: this researcher paved the way for a game-changing vaccine

- The cancer physician who helped to deliver a life-extending treatment

- ChatGPT and science: the AI system was a force in 2023 — for good and bad

NEWS 12 December 2023

More than 10,000 research papers were retracted in 2023 — a new record

The number of articles being retracted rose sharply this year. Integrity experts say that this is only the tip of the iceberg.

By

Richard Van Noorden

Retractions are skyrocketing as publishers work to remove sham articles from the literature.Credit: Klaus Ohlenschläger/Getty

The number of retractions issued for research articles in 2023 has passed 10,000 — smashing annual records — as publishers struggle to clean up a slew of sham papers and peer-review fraud. Among large research-producing nations, Saudi Arabia, Pakistan, Russia and China have the highest retraction rates over the past two decades, a Nature analysis has found.

The bulk of 2023’s retractions were from journals owned by Hindawi, a London-based subsidiary of the publisher Wiley (see ‘A bumper year for retractions’). So far this year, Hindawi journals have pulled more than 8,000 articles, citing factors such as “concerns that the peer review process has been compromised” and “systematic manipulation of the publication and peer-review process”, after investigations prompted by internal editors and by research-integrity sleuths who raised questions about incoherent text and irrelevant references in thousands of papers.

Most of the Hindawi retractions are from special issues: collections of articles that are often overseen by guest editors and that have become notorious for being exploited by scammers to rapidly publish low-quality or sham papers.

On 6 December, Wiley announced on an earnings call that it would stop using the Hindawi brand name altogether, having previously shuttered four Hindawi titles and, towards the end of 2022, temporarily paused special-issue publication. Wiley will fold existing titles back into its own brand. As a result of the problems, said Wiley’s interim chief executive, Matthew Kissner, the publisher expects to lose out on between US$35 million and $40 million in revenue this fiscal year.

A Wiley spokesperson said that the publisher anticipated further retractions — they did not say how many — but that the company takes the view that “special issues continue to play a valuable role in serving the research community”. The spokesperson added that Wiley had put in place more rigorous processes to confirm the identity of guest editors and oversee manuscripts, removed ‘hundreds’ of bad actors — some of whom had held guest editor roles — from its systems, and scaled up its research-integrity team. It is also “pursuing legal means” to share data about the bad actors with other publishers and providers of tools and databases.

Hindawi’s retracted papers might have been mostly sham articles, but they were still collectively cited more than 35,000 times, says Guillaume Cabanac, a computer scientist at the University of Toulouse in France who tracks problems in papers, including ‘tortured phrases’ — strange wording choices used in efforts to evade plagiarism detectors — and signs of undisclosed use of artificial intelligence. “These problematic papers get cited,” he says.

Retractions are rising at a rate that outstrips the growth of scientific papers (see ‘Rising retraction rates’), and this year’s deluge means that the total number of retractions issued so far has passed 50,000. Although analyses have previously shown that the majority of retractions are due to misconduct, this is not always the case: some are led by authors who discover honest errors in their work.

The world’s largest database to track retractions, collated by the media organization Retraction Watch, does not yet include all of 2023’s withdrawn papers. To analyse trends, Nature combined the roughly 45,000 retractions detailed in that data set — which in September was acquired for public distribution by Crossref, a non-profit organization that indexes publishing data — with another 5,000 retractions from Hindawi and other publishers, with the aid of the Dimensions database.

Rising rates

Nature’s analysis suggests that the retraction rate — the proportion of papers published in any given year that go on to be retracted — has more than trebled in the past decade. In 2022, it exceeded 0.2%.

Among countries that have published more than 100,000 articles in the past two decades, Nature’s analysis suggests that Saudi Arabia has the highest retraction rate, of 30 per 10,000 articles, excluding retractions based on conference papers. (This analysis counts an article for a country if at least one co-author has an affiliation in that country.) If conference papers are included, withdrawals from the Institute of Electrical and Electronics Engineers (IEEE) in New York City put China in the lead, with a retraction rate above 30 per 10,000 articles (see ‘Countries with highest retraction rates’).

The analysis shows that around one-quarter of the total number of retractions are conference papers — and the bulk of those comprise withdrawals by the IEEE, which has pulled more than 10,000 such papers in the past two decades. The IEEE was the publisher with the highest number of retractions. It does not record when it retracts papers, but most of those removed were published between 2010 and 2011.

Preventive measures

Monika Stickel, director of corporate communications at the IEEE, says that the institute thinks its preventive measures and efforts identify almost all submitted papers that do not meet the organization’s standards.

However, Cabanac and Kendra Albert, a technology lawyer at Harvard Law School in Cambridge, Massachusetts, have found issues, including tortured phrases, citation fraud and plagiarism, in hundreds of IEEE papers published in the past few years, Retraction Watch reported earlier this year. Stickel says that the IEEE has evaluated those papers and found fewer than 60 that didn’t conform to its publication standards, with 39 retracted so far.

The 50,000 or so retractions recorded around the world thus far are only the tip of the iceberg of work that should be retracted, integrity sleuths say. The number of articles produced by ‘paper mills’ — businesses that sell bogus work and authorships to scientists — is estimated to be in the hundreds of thousands alone, quite apart from genuine papers that might be scientifically flawed. “Paper-mill products are a problem even if no-one reads them, because they get aggregated with others into review articles and laundered into the mainstream literature,” says David Bimler, a New Zealand-based research-integrity sleuth also known by the pseudonym Smut Clyde.

Nature 624, 479-481 (2023)

doi: https://doi.org/10.1038/d41586-023-03974-8

Reprints and permissions

NEWS 11 December 2023

‘Biocomputer’ combines lab-grown brain tissue with electronic hardware

A system that integrates brain cells into a hybrid machine can recognize voices.

By

Lilly Tozer

Part of a brain organoid, in which stem cells (pink) are differentiating into neurons (purple).Credit: Steve Gschmeissner/Science Photo Library

Researchers have built a hybrid biocomputer — combining laboratory-grown human brain tissue with conventional electronic circuits — that can complete tasks such as voice recognition.

The technology, described on 11 December in Nature Electronics[1], could one day be integrated into artificial intelligence (AI) systems, or form the basis of improved models of the brain in neuroscience research.

The researchers call the system Brainoware. It uses brain organoids — bundles of tissue-mimicking human cells that are used in research to model organs. Organoids are made from stem cells capable of specializing into different types of cell. In this case, they were morphed into neurons, akin to those found in our brains.

The research aims to build “a bridge between AI and organoids”, says study co-author Feng Guo, a bioengineer at the University of Indiana Bloomington. Some AI systems rely on a web of interconnected nodes, known as a neural network, in a way similar to how the brain functions. “We wanted to ask the question of whether we can leverage the biological neural network within the brain organoid for computing,” he says.

Harnessing brainpower

To make Brainoware, researchers placed a single organoid onto a plate containing thousands of electrodes, to connect the brain tissue to electric circuits. They then converted the input information into a pattern of electric pulses, and delivered it to the organoid. The tissue’s response was picked up by a sensor and decoded using a machine-learning algorithm.

To test Brainoware’s capabilities, the team used the technique for voice recognition by training the system on 240 recordings of 8 people speaking. The organoid generated a different pattern of neural activity in response to each voice. The AI learnt to interpret these responses to identify the speaker, with an accuracy of 78%.

Although more research is needed, the study confirms some key theoretical ideas that could eventually make a biological computer possible, says Lena Smirnova, a developmental neuroscientist at Johns Hopkins University in Baltimore, Maryland. Previous experiments have shown only 2D cultures of neuron cells to be able to perform similar computational tasks; this is the first time it has been shown in a 3D brain organoid.

Better brain model

Combining organoids and circuits could allow researchers to leverage the speed and energy efficiency of human brains for AI, says Guo.

The technology could also be used to study the brain, says Arti Ahluwalia, a biomedical engineer at the University of Pisa in Italy, because brain organoids can replicate the architecture and function of a working brain in ways that simple cell cultures cannot. There is potential to use Brainoware to model and study neurological disorders, such as Alzheimer’s disease. It could also be used to test the effects and toxicities of different treatments. “That’s where the promise is; using these to one day hopefully replace animal models of the brain,” says Ahluwalia.

But using living cells for computing is not without its problems. One big issue is how to keep the organoids alive. The cells must be grown and maintained in incubators, something that will be harder the bigger the organoids get. And more complex tasks will demand larger ‘brains’, says Smirnova.

To build upon Brainoware’s capabilities, Guo says that the next steps include investigating whether and how brain organoids can be adapted to complete more complex tasks, and engineering them to be more stable and reliable than they are now. This will be crucial if they are to be incorporated into the silicon microchips currently used in AI computing, he says.

Nature 624, 481 (2023)

doi: https://doi.org/10.1038/d41586-023-03975-7

References

1. Cai, H. et al. Nature Electron.https://doi.org/10.1038/s41928-023-01069-w (2023).

Reprints and permissions

NEWS 15 December 2023

How a bullying scandal closed a historic astronomy department

Lund University in Sweden reorganizes in the face of years of accusations against two professors.

By

Alexandra Witze

The former astronomy building at Lund University in Sweden.Credit: Lund Observatory

At Lund University in Sweden this week, astronomers moved out of a building that was custom-built to hold telescopes and other artefacts from their 350 years of history, and relocated to a physics building down the road. That’s because the astronomy department no longer exists, having been dissolved in the wake of a bullying scandal.

For more than three years, Lund University administrators have struggled to respond to numerous complaints that were filed against two senior astronomy professors. The university ultimately decided to assign the two professors to other departments and to disband the department of astronomy and theoretical physics, subsuming it as a division of the physics department.

Many astronomers at Lund have told Nature that because the university took so long to find a solution, it has been nearly impossible to do science at times during the past few years. The turmoil contributed to several prominent astronomers leaving the university. Other leading scientists remain, but the forced reorganization has disrupted their careers and research. “This is an insane situation,” says an astronomer who asked to remain anonymous because they are still at the university.

Sven Lidin, the dean of science at Lund University who oversaw the changes, says that he saw no other way forward. “My attempts to resolve the issues within the existing organization were unsuccessful, and, as a last resort, I decided that a reorganization was the only option left,” he wrote in an e-mail to Nature.

The reorganization has also upended the theoretical physics group at Lund, which had been part of the department of astronomy and theoretical physics and has now seen its members, too, dispersed among other departments. “It was not only the astronomers that were affected by this — the whole of theoretical physics has now been broken apart," says Leif Lönnblad, a theoretical physicist at the university.

Other institutions have taken the extreme step of reorganizing in the wake of a bullying scandal. In 2017, for instance, the Swiss Federal Institute of Technology Zurich merged an astronomy institute with a physics institute — but excluded an astronomy professor accused of bullying.

At Lund, the situation is striking for how long it has dragged on. Complaints against the two professors, Sofia Feltzing and Melvyn Davies, date back to at least 2008 at Lund Observatory — the name used to refer to astronomy research at Lund University. A May 2020 employee survey revealed the long-running tensions: it found that 70% of respondents at the observatory had observed harassment and bullying in their workplace, although it didn’t name the people allegedly responsible. A June 2021 article in Nature described the findings of two university-commissioned investigations that Feltzing and Davies had victimized, discriminated against or bullied colleagues.

Neither Feltzing nor Davies responded to Nature’s requests for comment.

A lengthy resolution

Soon after the 2020 employee survey was conducted, various groups of senior academics and graduate students began appealing to the university to take action.

A ‘meridian’ telescope that was used by Sweden’s first female astronomer is on display at Lund’s former astronomy building. As researchers move out of the building, it’s unclear what will happen to it.Credit: Lund Observatory

By 2021, Lidin had brought in an administrator from outside the astronomy department to take charge of the observatory; another one was brought in for 2022. Both spent months interviewing observatory employees and speaking to the labour unions representing Feltzing, Davies and others, who were involved because of Swedish workplace rules. But neither could apparently reach a resolution.

In August 2022 — more than two years after the problem became well known — Lidin told the department that “within the current organization, the deadlocks which exist are far too vast to be overcome in a credible and sustainable way”. Three months later, the university’s faculty board voted to dissolve the astronomy department, sending most of the astronomers to the physics department by 1 January 2023. Feltzing joined the university’s geology department, and Davies had earlier moved to the mathematics department.

Feltzing maintained an office in the astronomy building for more than a year after the scandal went public, with some restrictions on how often she could enter the building and how much notice she had to provide. “We were walking in the same corridors for the longest time,” says the astronomer who asked to remain anonymous. “It was just very strange.”

The longer a bullying situation drags on, “the more traumatic the situation becomes”, says Christina Björklund, a researcher who studies bullying at the Karolinska Institute in Stockholm. Recent research, including the first study of the prevalence of harassment in bullying in Swedish academia, has highlighted how universities need to develop effective responses, she says.

Many current and former members of Lund Observatory say that if the astronomy department had to be dissolved, the physics department was the most suitable place for most of them to relocate to. But the drawn-out nature of the reorganization took a huge toll. “It’s done a lot of damage,” says Paul McMillan, an astrophysicist who recently left Lund University for the University of Leicester, UK. “It’s going to take time to rebuild.”

“It is certainly true that valuable time was lost in making the necessary changes and that this affected teaching and science,” Lidin says.

Vacated space

Astronomy is a historically important field of research in Lund, a city whose first observatory was built in 1672. Today, the astronomy building on the university campus has telescopes on its roof and houses displays including a famous panorama of the Milky Way — compiled at Lund in the 1950s — and a ‘meridian’ telescope that was used by Sweden’s first female astronomer to measure stars crossing a north–south line in the sky.

Lund Observatory astronomers are taking some of its historical artefacts in the move to the physics building, but probably not all of them. Given this upheaval, the forced relocation represents “the destruction of a culture”, says Colin Carlile, a guest researcher at the observatory. The university has not decided what to do with the space vacated by the astronomers.

Among the careers affected is that of Rebecca Forsberg, who recently received her PhD at Lund Observatory. Senior astronomers were distracted by the workplace issues and had less time and attention to work with students, she says. And the university didn’t provide sufficient resources for students to navigate the situation: “They’ve really gone way and above to make everything hard for us,” she says. “I don’t feel like they have protected us in any way.” Forsberg plans to leave Lund Observatory and might leave academia.

Lidin says he cannot comment on individual employees but that “it is possible” that the turmoil at Lund Observatory “has affected career choices for some of our staff”. He says he and other university officials “have done our utmost” to provide support for staff and students alike, but that “given the situation, I am not entirely surprised that there are those who felt we could have done more”.

One high-profile departure is Anders Johansen, an expert in planetary formation who headed Lund’s astronomy department from 2016 to 2020 and so handled many of the complaints about Feltzing and Davies. Johansen says that he did not have support from those higher up in the university to deal promptly and effectively with complaints. “At some point it became unbearable,” he says. “I did not want a life where I had to deal with such problems all the time.” Johansen moved his research group to the University of Copenhagen, but he visits Lund’s astronomers once a week and will become a guest professor there.

Astrophysicist Florent Renaud, who filed a complaint against Feltzing and Davies in 2020, had been looking to leave Lund when he was offered a leading position at the University of Strasbourg in France, his home country. “I picked the earliest date I could” as a start date, he says. Now that he’s in Strasbourg, “it’s so bright, and it feels weird after all these years of darkness”.

Renaud says that he is clear-eyed about the prevalence of harassment and bullying in academia. “When I said I’m going to leave Lund, people said: ‘Good for you, but be careful because you find harassers everywhere.’ I said, ‘I know, but what is different is the way they are managed or handled.’”

In the coming weeks, the astronomers who remain at Lund will unpack their boxes in the physics building. Many, exhausted by the past few years, are trying to look forward. “My energy is probably best and most productively spent doing really good research and proving that astronomy in Lund is very important and relevant and has a future,” says another astronomer who requested anonymity because they are still at the university.

“We have a stronger-knit community because of this,” adds the first astronomer who requested anonymity. “But I would not hug the dean and say thank you.”

Nature 624, 482-483 (2023)

doi: https://doi.org/10.1038/d41586-023-03953-z

Reprints and permissions

NEWS 12 December 2023

‘It’s all gone’: CAR-T therapy forces autoimmune diseases into remission

Engineered immune cells, most commonly used to treat cancers, show their power against lupus and other immune disorders.

By

Heidi Ledford

T cells (smaller cells) can be engineered to recognize cancer cells — and also other immune cells.Credit: Steve Gschmeissner/Science Photo Library

Engineered immune cells have given 15 people with once-debilitating autoimmune disorders a new lease on life, free from fresh symptoms or treatments. The results raise hopes that the approach — called CAR-T-cell therapy — might one day be extended to a variety of other conditions fuelled by rogue immune cells that produce antibodies against the body’s own tissues.

All 15 participants, who each had one of three autoimmune conditions, have remained disease-free or nearly so since their treatment, according to data presented on 9 December at the American Society of Hematology meeting in San Diego, California. The first participants were treated more than two years ago.

These successes, although preliminary, have been electric, says Marco Ruella, an oncologist at the University of Pennsylvania in Philadelphia. “We’re all excited,” he says. “There’s a lot of potential.”

Bespoke immune cells

CAR-T therapies harness the immune players called T cells. T cells are removed from the person being treated, genetically engineered to produce proteins called chimeric antigen receptors (CARs) and then reintroduced to the person’s body. In many therapies, the T cells are tailored to recognize a protein made by immune cells called B cells. When reintroduced, the CAR T cells will target the B cells for destruction — a useful feature for treating cancers caused by abnormal B cells.

B cells also drive some autoimmune disorders by making antibodies that attack healthy tissue. In 2019, researchers showed that CAR T cells that recognize B cells reduced symptoms in mice with a disease similar to lupus, an autoimmune disorder that affects a variety of organs[1].

Around the same time, researchers at University Hospital Erlangen in Germany were setting up their own CAR-T centre to provide cancer treatment. During a meeting at the centre, a rheumatologist asked the cancer specialists for advice about a young woman with a form of lupus called systemic lupus erythematosus. Several of her organs were failing; her doctors estimated that she did not have long to live. The young woman insisted that they try something new.

High-risk approach

The team thought of the mouse study but baulked at trying it in people. CAR-T therapy can have severe side effects, and recipients must first undergo intensive chemotherapy that kills off many of their existing immune cells. “At the beginning we were quite scared,” said team member Fabian Müller, an oncologist at the Friedrich–Alexander University of Erlangen–Nuremberg, at a press conference before he presented the work at the San Diego meeting. The woman was adamant that they try.

That first participant — and the others who followed — experienced relatively minor adverse effects, Müller reported at the conference. The Erlangen team eventually used the method to treat two other autoimmune disorders: systemic sclerosis and idiopathic inflammatory myositis. The successes continued.

Other groups have since taken up the approach and reported similar results. Earlier this month, another team added a fourth autoimmune disorder called myasthenia gravis to the list of successes[2]. Researchers are beginning to wonder how long the final list will be. “We’re just at the beginning,” says Marcela Maus, who designs CAR-T therapies against cancer at Massachusetts General Hospital in Boston. “There is so much that can be done that was unthinkable just a decade ago.”

At this stage, however, it’s unclear how much of this success is due to the CAR-T therapy as opposed to the chemotherapy that killed many of the participants’ pre-existing immune cells, cautions Ruella. That might have helped to wipe out the errant B cells.

For now, Müller lapses into a dreamy smile as he marvels over the remarkable recoveries he has seen: the man who struggled to walk 10 metres before his treatment and now routinely walks 10 kilometres around town, for example. “These are young people that have been spending more time with their doctors than with their friends,” he says. “They would describe their breakfast as a handful of pills that they are just shoving in.”

“And it’s all gone,” he says. “From the physician perspective, it’s pretty much the most pleasing thing.”

Nature 624, 483-484 (2023)

doi: https://doi.org/10.1038/d41586-023-03968-6

References

1. Kansal, R. et al. Sci. Transl. Med. 11, eaav1648 (2019).

2. Haghikia, A. et al. Lancet Neurol. 22, 1104–1105 (2023).

Reprints and permissions

NEWS 13 December 2023

COP28 climate summit signals the end of fossil fuels — but is it enough?

As nations make historic pledge to ‘transition’ energy systems away from fossil fuels — some scientists are disappointed by the softened wording.

By

Katharine Sanderson

Delegates applauding at the end of COP28 in Dubai.Credit: Fadel Dawod/Getty

Scientists have voiced mixed reactions to a pledge to “transition away from fossil fuels” made by the world’s governments at the end of the United Nations COP28 climate summit in Dubai.

“It’s major,” says Lisa Schipper a developmental geographer at the University of Bonn, Germany. Previous end-of-COP declarations have failed to mention fossil fuels in this way. At COP26 in Glasgow, delegates pledged to “phase down” coal use without carbon capture and storage, which was regarded as a first at the time.

COP28 is the first ‘global stocktake’ of progress towards meeting the goals of the 2015 Paris Climate Agreement (COP21), in which representatives of more than 190 countries pledged to limit global warming to within 1.5 ºC of pre-industrial levels.

The final COP28 text calls on parties to be “transitioning away from fossil fuels in energy systems, in a just, orderly and equitable manner, accelerating action in this critical decade, so as to achieve net zero by 2050 in keeping with the science”.

The text, which was agreed on the morning of 13 December after several all-night negotiating sessions, “shows that they are actually listening to the science”, Schipper says. But she adds that “transitioning away”, rather than “phasing out”, fossil fuels is nevertheless disappointing, because ‘transition’ could be interpreted in different ways. “It doesn’t mean eliminating, whereas ‘phasing out’ fossil fuels, is about ‘the end’,” she says.

The UN Framework Convention on Climate Change, the agency that organizes COP meetings, stated at the top of its press release that the COP28 agreement “signals the beginning of the end of the fossil fuel era”. However, hours after the agreement, Saudi Arabia’s energy minister, Abdulaziz bin Salman, was reported in Al Arabiya, as saying that it would not affect the country’s exports of crude oil.

Mizan Khan, an environmental scientist at the International Centre for Climate Change and Development in Dhaka, Bangladesh, says that most climate-vulnerable countries, including Bangladesh, wanted, at a minimum, to see in the agreement language on phasing out fossil fuels. But he adds that COP meetings rarely produce strong outcomes, because decisions are made through a consensus of more than 190 countries.

The requirement under UN rules that countries agree unanimously on text is a “fatal flaw” in the COP process, says Charles Fletcher, a climate scientist at the University of Hawaiʻi at Mānoa who studies sea-level rise. At COP28, former US vice-president Al Gore, a long-time environmental campaigner, urged member states to consider taking decisions agreed on by only 75% of nations. “It’s almost ludicrous that we are asking the leaders of fossil-fuel production to shepherd humanity into a safe climate future,” Fletcher adds.

Strength in numbers

The text agreed on 13 December stands in contrast to a version released on Monday 11 December that called for “reducing both consumption and production of fossil fuels”. That was criticized as not being strong enough, in a rare sign of unity from the representatives from different parts of the globe.

However, delegates from the Alliance of Small Island States (AOSIS) — which are among the most vulnerable to climate change — said in a statement that they weren’t in the room when the final deal was agreed. “We were working hard to coordinate the 39 small island developing states that are disproportionately affected by climate change, and so were delayed in coming here.” AOSIS is advocating a target that would see global emissions peak by 2025.

“It is not enough for us to reference the science and then make agreements that ignore what the science is telling us we need to do. This is not an approach that we should be asked to defend,” its statement says.

Follow the money

This year’s COP meeting started optimistically, with another historic deal in which rich countries pledged more than US$700 million to a new ‘loss and damage fund’ to support those nations that are most affected by climate change. Some 130 participating governments also pledged to treble renewable energy generation capacity by 2030.

The principle for a loss-and-damage fund had been agreed at COP27 last year in Egypt. “We started the COP with this huge bang,” says Schipper. But she adds that COP28 has otherwise been disappointing when it comes to promises on climate finance.

The agreement includes text on how to provide funds to help countries adapt to the impacts of climate change, but Schipper calls this “super-vague”. The meeting needed to set actual targets or make specific financial commitments, but it did not do this, she says. From a scientific perspective, Schipper describes the COP28 text on climate adaptation as a “very dangerous straitjacket for adaptation finance”.

Romain Weikmans, who studies climate finance at the Free University of Brussels, adds that “the big chunk of the climate finance” discussion will now take place next year at COP29, which will be in Baku, Azerbaijan.

Nature 624, 484-485 (2023)

doi: https://doi.org/10.1038/d41586-023-04025-y

Additional reporting by Gemma Conroy.

Reprints and permissions

NEWS 14 December 2023 Correction 14 December 2023

RNA biologist loses disability case against Howard Hughes Medical Institute

Vivian Cheung sued after her funding was not renewed, alleging discrimination, but the institute said her science no longer met its expectations.

By

Amanda Heidt &

Max Kozlov

Vivian Cheung filed a lawsuit against the Howard Hughes Medical Institute in 2020.Credit: Ricky Carioti/The Washington Post via Getty

Rockville, Maryland

After deliberating for just over three hours, a jury at a courthouse in Rockville, Maryland, has found that the Howard Hughes Medical Institute (HHMI) did not use the disability of former employee Vivian Cheung, a paediatric neurologist and RNA biologist, as a basis for discontinuing her funding. Researchers hoping to shine a light on disability in the scientific workplace say they are disappointed with the outcome, given that US disability law has lagged behind legislation for other types of discrimination.

For its part, the “HHMI is pleased that a Montgomery County jury has agreed that Dr. Cheung’s allegations are without merit”, Erin O’Shea, president of HHMI in Chevy Chase, Maryland, said in a statement. “We look forward to putting this matter behind us.”

“This goes to show that the power of large private science institutions and peer-review systems are hard to challenge,” David Oppenheimer, director of the Berkeley Center on Comparative Equality and Anti-Discrimination Law in California and Cheung’s attorney, told Nature. “Vivian Cheung is a remarkably courageous person to take on such a powerful institution.”

Cheung, who studies rare genetic diseases at the University of Michigan in Ann Arbor, was herself diagnosed in 2014 with a rare disorder that impaired her vision and mobility. During the eight-day trial, Cheung’s legal team alleged that her disability spurred the HHMI not to renew her funding in 2018 and that she was entitled to more than US$2.7 million in compensation stemming from lost wages and the emotional toll of reputational damage.

The HHMI, which is one of the world’s largest private funders of fundamental biomedical research and supports about 260 scientists across the United States through its flagship Investigator Program, denied that Cheung’s disability was tied to its decision. Instead, it argued that her funding was rescinded because her research no longer met the high calibre expected of its investigators.

This verdict is particularly noteworthy, because few legal claims involving disability result in a trial — much less with an institution as high profile as the HHMI. But the fact that it was ultimately unsuccessful could have a chilling effect on future cases, says Nathan Tilton, a disabled veteran and manager of the Disability Lab at the University of California, Berkeley. “I could absolutely see a case like this dissuading other disabled people from bringing their claims forward.”

In the courtroom

Cheung first received one of HHMI’s prestigious investigator awards in 2008, on the basis of her research identifying previously unseen differences between DNA and RNA. She used the HHMI funds to expand her investigations into DNA–RNA hybrid structures, called R-loops, that regulate gene expression, among other things. When her award was first renewed, in 2012, reviewers said that in future Cheung should move beyond observations of R-loops and towards a deeper understanding of how they work, she told Nature.

Cheung’s legal team introduced witnesses who testified that her research has done that. One of these witnesses was Bonnie Woolston, whose family has a rare, inherited form of motor neuron disease (amyotrophic lateral sclerosis), which leads to progressive loss of muscle control over time. Cheung’s research into the Woolston family has shown[1] that mutations in a single gene, called senataxin, lead to fewer R-loops in their cells, which in turn increase activity in a signalling pathway that has been linked to muscular disorders. Woolston said that a natural-history study that Cheung helped to establish at the US National Institutes of Health (NIH) remains one of the only existing efforts to study the disease in depth.

Heading into her second renewal process, in 2018, Cheung told Nature she felt that her research was even stronger than in 2012. But Cheung later testified that one witness, HHMI senior scientific officer Philip Perlman, asked her directly about her condition in a phone call about her upcoming renewal. She said that his comments had made her uncomfortable, noting that “even if my best friend asked me about my health and I told her ‘I don’t feel like talking about it’, I think she would stop. We were talking about my renewal”.

During his turn on the stand, Perlman said that he had written about having his “fingers crossed” that Cheung would take a medical phase-out in e-mails between himself and senior HHMI leadership, and that he had “probably shared too much information” about Cheung’s medical condition with a member of her review committee. But neither disclosure amounted to discrimination, the jury found, and witnesses for the defence denied that Cheung’s disability had played into their decision to give her scores that were among the lowest in the cohort of investigators undergoing renewal at the time.

Mary Beckerle, a cell biologist at the University of Utah in Salt Lake City, who observed both of Cheung’s renewal processes, stated that she had “never heard scientific leadership say anything ... that wasn’t related to the science”. O’Shea, who was at the time the HHMI’s vice-president and chief scientific officer and made the final determination of Cheung’s renewal, says that the consensus was unanimous: “The very clear recommendation made to me was not to renew her.”

What comes next

After Cheung lost her HHMI funding, her laboratory underwent a shake-up, she told Nature. She had to let some workers go, and others left on their own. Still, Cheung remains committed to furthering the work she began with the HHMI.

She has built funding back though grants from the NIH, and in April this year, she was awarded $2.3 million by the Warren Alpert Foundation, a philanthropic organization based in Providence, Rhode Island, seeking cures for medical conditions, to launch a ‘Human RNome’ project. Similar to the Human Genome Project, it will map the RNA of all human cells. “We know that RNA is going to be an important component in the future of medicine,” Cheung says.

Even though the jury found the HHMI was not liable for any damages to Cheung, observers hope that the case will serve as a wake-up call for funders. In 2021, the HHMI pledged $2 billion over 10 years to improve racial, ethnic and gender diversity in science, but made no mention of disability status.

Audrey Winkelsas, a medical student at the University of Michigan who has spinal muscular atrophy and has been mentored by Cheung, told Nature that “it is important for institutions to acknowledge disability ... and follow that up with tangible actions” so that scientists “can focus on their work and be as productive as possible”.

Nature 624, 485-486 (2023)

doi: https://doi.org/10.1038/d41586-023-04004-3

Updates & Corrections

- Correction 14 December 2023: An earlier version of this story stated that HHMI supports about 260 scientists across the United States. In general, it supports more scientists than that. The 260 scientists are specifically part of its flagship investigator programme.

References

1. Grunseich, C. et al. Mol. Cell 69, 426–437 (2018).

Reprints and permissions

The best science images of 2023 – Nature’s picks

Cosmic dust, microscopic syrup, a flying gecko and more.

Emma Stoye, Nisha Gaind, Katharine Sanderson and Carissa Wong

11 December 2023

Credit: NASA, ESA, CSA, STScI, Klaus Pontoppidan (STScI) Image Processing: Alyssa Pagan (STScI)

Credit: NASA, ESA, CSA, STScI, Klaus Pontoppidan (STScI) Image Processing: Alyssa Pagan (STScI)

Images selected by Nature’s visuals team, text by Emma Stoye, Nisha Gaind, Katharine Sanderson and Carissa Wong

NASA’s James Webb Space Telescope continued to dazzle in 2023, with some of the most spectacular images of space ever seen. Closer to home, photographers and researchers captured unknown species and hidden microscopic scenes. From cosmic dust to flying geckos, here are the images that caught the eyes of Nature’s editors.

Credit: Dave Sanders/New York Times/Redux/eyevine

Credit: Dave Sanders/New York Times/Redux/eyevine

Orange apocalypse. In June, the northeastern United States was choked by smoke blown down from Canada’s extreme wildfires, which turned the sky a disturbing orange, seen here at New York City’s Brooklyn Bridge. Scientists say that climate change is driving increasingly hot, dry and windy weather.

Credit: Thomas Vijayan

Credit: Thomas Vijayan

Melt warning. This shot of melt water pouring through the Austfonna ice cap on the Arctic island of Nordaustlandet, Norway, won the Nature category in the 2023 Drone Photo Awards. “I have visited this place several times before, but last year it was disheartening to witness the sea ice melting as early as June,” said photographer Thomas Vijayan.

Credit: Xinhua/Shutterstock

Credit: Xinhua/Shutterstock

Ancient flex. This microfossil — about 2 millimetres across — shows a rare example of preserved muscle structures in early animals. The specimen is about 535 million years old and belongs to the cycloneuralia, the group of animals that includes roundworms. They were found in China’s Shaanxi province. The muscles would have assisted movement and feeding, say palaeontologists.

Sun serpent. This huge, snake-like solar filament was captured by astrophotographer Eduardo Schaberger Poupeau, and won the Our Sun category of the 2023 Astronomy Photographer of the Year competition. Filaments are made of plasma that sticks out from the Sun’s surface, shaped by magnetic fields.

Credit: Eduardo Schaberger Poupeau

Credit: Eduardo Schaberger Poupeau

Seed spreader. Drosophila fruit flies have some of nature’s longest sperm at 2 millimetres. This image of fly sperm growing in the testis was a winner in the 2023 MIT Koch Institute Image Awards. Each mature sperm (blue) starts as a stem cell (top) then elongates. Cell nuclei are white. Magenta and yellow show expression of RNA essential for sperm  development.

Credit: Jaclyn Fingerhut, Yukiko Yamashita - MIT Department of Biology, Whitehead Institute

Credit: Jaclyn Fingerhut, Yukiko Yamashita - MIT Department of Biology, Whitehead Institute

Space Monet. This awe-inspiring image by the James Webb Space Telescope shows stars forming in the Rho Ophiuchi cloud complex, the closest place to Earth where stars are being born. Its ethereal texture evokes that of impressionist paintings. Jets of hydrogen gas (red) spurt from young stars, illuminating the interstellar gas. The glowing ‘cave’ below is formed by stellar winds blowing out of a young star.

Credit: NASA, ESA, CSA, STScI, Klaus Pontoppidan (STScI) Image Processing: Alyssa Pagan (STScI)

Credit: NASA, ESA, CSA, STScI, Klaus Pontoppidan (STScI) Image Processing: Alyssa Pagan (STScI)

Credit: Lal Muansanga

Credit: Lal Muansanga

Hello, gecko! This year, scientists discovered a new species of flying gecko, Gekko mizoramensis. The lizards live in Mizoram state in northeast India. They use the wing-like flaps of skin on their legs and feet to glide through the forest, from tree to tree. The find shows how little is known about plant life and animals in the area — there could be many more species unknown to science lurking there.

Credit: USGS/Handout via REUTERS

Credit: USGS/Handout via REUTERS

Lava monster. Hawaii’s most active volcano, Kīlauea, erupted in June, creating a pool of lava in the Halema’uma’u crater. The volcano is studded with cameras and instruments that measure ground deformation and seismic activity.

Yeast snowflake. This starry cluster of yeast cells is the result of experiments that probed the evolution of multicellular organisms. Researchers selected large cells that evolved to form branching clumps.

Credit: Anthony J. Burnetti

Credit: Anthony J. Burnetti

Credit: NASA/JPL-Caltech/SwRI/MSSS/Brian Swift

Credit: NASA/JPL-Caltech/SwRI/MSSS/Brian Swift

Swirling cyclones. During its 54th orbit of Jupiter, NASA’s Juno spacecraft captured close-up views of the storms that surround the planet’s north pole. Citizen scientist Brian Swift processed one image to produce this high-contrast version, which highlights the cyclones.

Credit: Jialing Cai/Ocean Photographer of the Year

Credit: Jialing Cai/Ocean Photographer of the Year

Stick drift. This enchanting paper nautilus octopus (Argonauta sp.) is hitching a ride on a stick in the dark of the Pacific Ocean off the Philippines. It is surrounded by sediment from a volcanic eruption that twinkles in the camera’s light.

Credit: Dr. Diego García/Courtesy of Nikon Small World

Credit: Dr. Diego García/Courtesy of Nikon Small World

Syrup’s sharp edges. Sugar syrup isn’t always oozy. Crystallized, seen under a polarized-light microscope and magnified 25 times, the substance’s spiky, layered structure is revealed. This counter-intuitive image was showcased in the 2023 Nikon Small World Photomicrography Competition.

Credit: Relativity/John Kraus

Credit: Relativity/John Kraus

Print-a-rocket. The first launch of a rocket made from 3D-printed parts lit up the night sky above Florida in March. Terran 1 was built by aerospace company Relativity Space in Long Beach, California.

A PERSONAL VIEW OF THE NEWS

In compiling this year’s collection of striking science images, Nature’s media editors each identified a photograph that said something special to them. Here is their take on the past 12 months.

Uncharted waters. Agnese Abrusci (Media editor). This shot of a shipwreck stranded on Namibia’s treacherous Skeleton Coast is among the winners of the 2023 Astronomy Photographer of the Year competition. The vessel seems to ride the fog, lit by star trails shining through the cloudy sky. The photo conveys some of the unsettling feelings we have all experienced over the past year. Amid earthquakes, floods, war, widening inequalities and an escalating climate emergency, it is tempting to hide in a comforting emotional ‘fog’ away from the world. Whether we will rise from the haze, or sink into it, remains to be seen.

Sole survivor. Michael Szebor (Locum media editor). A lone house remains standing in Lahaina, Hawaii, surrounded by the charred remains of the rest of its neighbourhood. With its volcanic landscapes, Hawaii is no stranger to natural disasters, but wildfires on the island of Maui earlier this year took many people by surprise. This picture brings home the scale of the devastation in hard-hit urban areas. It is thought that recent renovations helped this house to survive, while others around it were burnt to the ground.

Junk food. Amelia Hennighausen (US media editor). As Earth’s natural resources dwindle, wild species must find new ways to survive. These elephants in Ampara, Sri Lanka, are forced to forage in rubbish dumps because so much of their habitat has been lost. They and other animals can become ill or die if they ingest too much plastic. The country has banned some single use plastics, but without enough natural habitat, human–elephant conflicts near wildlife reserves will continue.

Wildfire aftermath. Tom Houghton (Locum managing media editor). Photographer Alkis Konstantinidis captured this moment of compassion as wildfires swept through the Greek village of Hasia in August. Amid the haze and smoke of the raging blaze, two volunteers hand feed water to a sheep rescued from a burning farm. I was drawn to the painterly framing of this powerful shot.

Coral reflections. Jessica Hallett (Associate media editor). This stunning image of corals mirrored at low tide came third in the Conservation (Hope) category of this year’s Ocean Photographer of the Year competition. After researching many coral-bleaching images for stories this year, it was a breath of fresh air for me to see such a beautifully pristine reef photographed in such spectacular fashion. As a keen snorkeller and diver, I can appreciate the difficulty in keeping the water as still as possible to get the perfect reflection.

Springer Nature

- Privacy Policy

- Use of cookies

- Legal notice

- Terms & Conditions

- Accessibility statement

© 2023 Springer Nature Limited. All rights reserved.

TopBuilt with Shorthand

Nature’s 10

Ten people (and one non-human) who helped shape science in 2023

13 DECEMBER 2023

Image: Kim Albrecht; artificial-worldviews.kimalbrecht.com

Image: Kim Albrecht; artificial-worldviews.kimalbrecht.com

An AI pioneer, an architect of India’s Moon mission and the world’s first global heat officer are some of the people behind this year’s big stories.

The Nature’s 10 list explores key developments in science over the past year and some of the individuals who helped to make amazing discoveries and bring attention to crucial issues. It is not an award or a ranking, but a selection compiled by Nature’s editors to highlight the year’s most compelling stories.

Since its inception more than a decade ago, Nature’s 10 has highlighted the influence of 10 people within the world of science. We are continuing with that tradition in 2023 and are adding to it by including a non-person – an acknowledgement of the role that artificial intelligence designed to mimic human language is having in the development and progress of science.

Kalpana Kalahasti: To the Moon

This engineer and manager played a crucial part in ensuring Chandrayaan-3’s triumphant touchdown on the Moon, making India only the fourth country to achieve the feat.

Read Kalpana Kalahasti’s full profile

Marina Silva: Amazon protector

Brazil’s minister of the environment helped to rein in rampant deforestation and rebuild institutions that were weakened by the previous government.

Read Marina Silva’s full profile

Katsuhiko Hayashi: Rewiring reproduction

His feat of creating viable eggs from the cells of male mice could help to save species on the brink of extinction.

Read Katsuhiko Hayashi’s full profile

Annie Kritcher: Fusion igniter

This physicist helped the US National Ignition Facility to produce nuclear reactions once seen only in hydrogen bombs and stars.

Read Annie Kritcher’s full profile

Eleni Myrivili: Warming warden

As the United Nations chief heat officer, this former politician is helping the world to prepare for the threats of climate change.

Read Eleni Myrivili’s full profile

Ilya Sutskever: AI visionary

A pioneer of ChatGPT and other AI systems that are changing society.

Read Ilya Sutskever’s full profile

James Hamlin: Superconductivity sleuth

This physicist helped to spot flaws in sensational claims of superconductivity at room temperature.

Read James Hamlin’s full profile

Svetlana Mojsov: Unsung drug developer

How a biochemist finally gained recognition for her part in developing multibillion-dollar weight-loss drugs.

Read Svetlana Mojsov’s full profile

Halidou Tinto: Malaria fighter

A second vaccine for a deadly scourge will soon roll out, thanks to this researcher’s rigorous testing.

Read Halidou Tinto’s full profile

Thomas Powles: Cancer explorer

This physician and cancer researcher led a transformative clinical trial for the treatment of severe bladder cancer.

Read Thomas Powles’s full profile

ChatGPT: Boon and burden?

The poster child for generative AI software represents a potential new era for science.

Read ChatGPT’s full profile

Ones to watch in 2024

- Monica M. Bertagnolli: Director, US National Institutes of Health

This surgical oncologist will steer the world’s largest funder of biomedical research, which was without an officially confirmed director for nearly two years.

- Colin Waters: Chair, Anthropocene Working Group

A geologist at the University of Leicester, UK, Waters heads a research committee seeking to define the Anthropocene geological epoch.

- Ilan Gur: Chief executive, UK Advanced Research and Invention Agency

This materials scientist will oversee the United Kingdom’s push to fund high-risk, high-reward research.

- Muhammad Masroor Alam: Molecular biologist, Pakistan National Institutes of Health

The director of Pakistan’s national polio lab leads the World Health Organization’s surveillance efforts in the country, a key role in the push to eliminate wild poliovirus there.

Related articles

The best science images of 2023 – Nature’s picks

Science and the new age of AI

Springer Nature

- Privacy Policy

- Use of cookies

- Legal notice

- Terms & Conditions

- Accessibility statement

© 2023 Springer Nature Limited. All rights reserved.

TopBuilt with Shorthand

NEWS FEATURE 13 December 2023

The engineer who helped India to reach the Moon

Kalpana Kalahasti had a crucial role in ensuring Chandrayaan-3’s triumphant touchdown on the Moon.

By

Jatan Mehta

Credit: Samyukta Lakshmi for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

“We have achieved our goal flawlessly,” said Kalpana Kalahasti, a few minutes after India’s space agency safely landed its first probe on the Moon on 23 August, part of the Chandrayaan-3 mission. “This will remain the most memorable and happiest moment for all of us.”

The successful mission by the Indian Space Research Organisation (ISRO) put the country in a small group that has managed to set a craft down on the lunar surface. Only the Soviet Union, the United States and China have also achieved the feat. And as associate project director of Chandrayaan-3, Kalahasti played a crucial part in ensuring its success.

Chandrayaan-3 carried with it the hopes and fears of a nation when it lifted off on 14 July. India’s previous attempt to reach the lunar surface, the Chandrayaan-2 mission in 2019, ended in failure when the lander crashed. Three other Moon missions had similar fates: the 2019 loss of the Beresheet lander built by Israeli company SpaceIL, and this year the crashes of HAKUTO-R Mission 1 from the Japanese company ispace and Russia’s Luna 25 lander.

The loss of Chandrayaan-2’s lander was a defining moment for Kalahasti and her team members, who poured all of their efforts into bouncing back. “From the day we started rebuilding our spacecraft after the Chandrayaan-2 experience, it has been breathe in, breathe out Chandrayaan-3 for the team,” she said after this year’s landing.

One of the biggest challenges the team faced was that the total mass of — and available budget for — the spacecraft had to remain the same as those for Chandrayaan-2. That meant the team could not drastically redesign the lander or build in many redundancies. So Kalahasti worked with project director Palanivel Veeramuthuvel to reconfigure the Chandrayaan-2 mission’s orbiter and lander. ISRO reduced the mass of the orbiter to provide the lander with extra fuel, stronger legs and other improvements.

“This is where Chandrayaan-2’s flight was invaluable. Its many systems that did work allowed us to arrive at an optimum Chandrayaan-3 configuration,” Kalahasti told Nature.

Veeramuthuvel and Kalahasti spent the bulk of Chandrayaan-3’s development time devising and overseeing comprehensive tests and simulations, such as assessing the navigation system’s ability to avoid hazards before touchdown on Moon-like terrain.

“The goal was to have a well-documented, well-understood system. There was no compromise in demonstrating the system’s performance,” says Kalahasti.

The efforts paid off. But conducting so many tests and integrating their results while also planning the flight was a giant task that required coordinating a dozen ISRO centres across the country. “It was as if we were building five to six different satellites together,” says Kalahasti. She relied on her past experiences in project management and systems engineering, including her leading roles in the development of several ISRO’s Earth-observation satellites.

Her leadership role on the Moon mission was a long way from her beginnings at ISRO in 2000. She was drawn to the agency, she says, by a desire to work at a core engineering organization that would leverage her degree in electronics and communications. Her first job with the agency was as a radar engineer at the Satish Dhawan Space Centre in Sriharikota, where ISRO launches its missions.

Kalahasti is elated that the Chandrayaan-3 mission has ignited a spark among younger people in India. “Apart from the mission’s technical aspects, I hope young professionals across India and the world get inspired by how the team meticulously emerged from failure.”

The mission’s success has inspired confidence in other nations and companies hoping to attempt future Moon landings, says Jessy Kate Schingler, a space-policy researcher and senior adviser at the Open Lunar Foundation, a non-profit organization in San Francisco, California, that is advocating for a peaceful lunar presence. “It’s really nice to see India coming back for a second try on this mission soon after its first attempt,” she says. “It’s such a hard thing to do, a Moon landing. So Chandrayaan-3, I think, is an appreciated investment the whole world will benefit from.”

Kalahasti is excited about what ISRO could take on next. The agency wants to send a mission to retrieve lunar samples, as a precursor to its 2040 goal of landing a human crew on the Moon. “Now that the critical aspect of demonstrating a Moon landing is done,” says Kalahasti, “we can move towards other capabilities.”

Nature 624, 496-497 (2023)

doi: https://doi.org/10.1038/d41586-023-03920-8

Reprints and permissions

NEWS FEATURE 13 December 2023

Amazon protector: the Brazilian politician who turned the tide on deforestation

As Brazil’s environment minister, Marina Silva helped to rein in rampant deforestation and rebuild institutions that were weakened by the previous government.

By

Meghie Rodrigues

Credit: Adriano Machado for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

In a year that brought unrelenting bad environmental news, with record global warming, searing heatwaves and fires, Marina Silva delivered a hopeful message on 3 August. Brazil’s environment and climate-change minister announced that there had been a 43% drop in deforestation alerts on the basis of satellite images of the Amazon rainforest between January and July 2023, compared with the same period in 2022. This was a sharp shift from the previous four years, which had seen a marked rise in such alerts.

The turnaround for environmental protections in Brazil started on 1 January, when Luiz Inácio Lula da Silva took office as president and Marina Silva assumed her current role. It’s her second time heading the ministry of the environment and climate change, which she ran previously between 2003 and 2008, during Lula da Silva’s first and second presidencies.

During her first time in office, Marina Silva tackled rampant forest-clearing activities by leading the development of the Action Plan for the Prevention and Control of Deforestation in the Legal Amazon (PPCDAm) — a programme that achieved an 83% decrease in deforestation between 2004 and 2012 in the Brazilian Amazon.

But many of the protections she helped to put in place were dismantled by the government of Jair Bolsonaro, Brazil’s president from 2019 to 2022. During his term, the government issued 40% fewer fines for environmental crimes, and logging in the Amazon increased by about 60% compared with the four previous years.

Silva and her team started this year, she says, “with the tough mission to reconstruct what had been dismantled and, at the same time, create new results for environmental policy”.

From an early age, Silva has embraced difficult challenges. She was born in 1958 in Rio Branco, Brazil, in the heart of the Amazon region. Coming from a poor family of 11 children (3 of whom died young), Silva started work at an early age along with her siblings, extracting latex from rubber trees. She wanted to be a nun and didn’t learn to read or write until she was a teenager.

Silva met environmental activist Chico Mendes (who was killed in 1988 by a rancher) on a course in rural leadership in the mid-1970s and started her career in environmental activism, which led eventually to politics. In 1994, she became Brazil’s youngest elected senator at 35 years old.

In her current role, Silva is not always in alignment with the current government, says Pedro Jacobi, an environmental-governance researcher at the University of São Paulo, Brazil. The Lula da Silva government intends to increase drilling for oil and gas — including at the mouth of the Amazon River, says Jacobi. So the environment ministry is “walking on thin ice all the time”, he says.

But in terms of controlling and preventing deforestation, Brazil is doing its homework, says Natalie Unterstell, president of the Talanoa Institute, a climate-policy organization based in Rio de Janeiro, Brazil. “Marina Silva’s leadership on this agenda is very important and she is doing an extraordinary job,” says Unterstell.

One key achievement was launching a revamped version, on 5 June, of the PPCDAm programme to protect the Amazon, which the Bolsonaro administration had shut down. Silva also reinstated support for policing the region to enforce environmental regulations. And it was soon clear that the policies were working. Between January and July, the Brazilian Institute of Environment and Renewable Natural Resources (IBAMA) issued 147% more fines for environmental crimes than it had averaged during similar months between 2019 and 2022.

According to data from Brazil’s National Institute for Space Research (INPE), deforestation in the Amazon from August 2022 to July 2023 is estimated to be 22% below what it was in the previous 12 months. The rate is the lowest since 2018, but is about twice that of 2012, when deforestation was the lowest since INPE satellites began taking measurements in 1988.

Silva says one of the keys reasons why environmental protections are working is that a wide swathe of the government is promoting this agenda. “What fills me with joy,” she says, “is to see at work a concept that is very dear to me — that environmental policy should not be restricted to only one sector, but traverse all ministries.”

But ending deforestation is not enough. “If countries do not reduce their CO_2 emissions from fossil fuels, forests run the risk of being destroyed due to climate change, in the same way. So we need a civilizational change, a change in our ways of life.”

She likens herself to a strong fibre from an Amazonian tree, which is used to bind wood to create rafts. “This is how I see my work,” she says, “bringing together those who are available and whatever is necessary to form a support surface in the challenging journeys of our time.”

Nature 624, 498 (2023)

doi: https://doi.org/10.1038/d41586-023-03921-7

Reprints and permissions

NEWS FEATURE 13 December 2023

Making mice with two dads: this biologist rewrote the rules on sexual reproduction

Katsuhiko Hayashi created viable mouse eggs from male cells, a feat that could help to save species on the brink of extinction.

By

Heidi Ledford

Credit: Kosuke Arakawa for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

When Katsuhiko Hayashi and his colleagues announced in March that they had produced mouse pups from the cells of two male parents, the news literally floored some researchers. “I fell off my chair,” says Robert Gilchrist, a reproductive biologist at the University of New South Wales in Sydney, Australia. “It’s an astonishing scientific achievement.”

It was the culmination of years of painstaking work by Hayashi, a developmental biologist now at Osaka University in Japan. He and his team had previously devised methods for coaxing mouse stem cells into becoming immature eggs, or oocytes, then maturing them, fertilizing them and producing live pups (O. Hikabe et al. Nature 539, 299–303; 2016). This year, Hayashi revealed that his laboratory had built on that success by using cells from male mice to produce the eggs (K. Murakami et al. Nature 615, 900–906; 2023). Many researchers had thought the task impossible, or nearly so.

Hayashi, however, is more reserved in assessing what he achieved: “Actually, it’s not that difficult.”

Hayashi and his colleagues took cells from the tails of male mice, which have both X and Y sex chromosomes, and converted them into stem cells. In the process, roughly 3% of such cells spontaneously lose their Y chromosomes. The team then isolated these Y-less cells and treated them with a chemical that causes errors during cell division.

Some of those errors resulted in cells with a duplicated X chromosome, effectively making them female cells. The team then took these through the complex and laborious process required to make an egg. Each step is delicate and many cells are typically lost, says Mitinori Saitou, a developmental biologist at Kyoto University in Japan who collaborates with Hayashi.

The team fertilized the eggs and transplanted the resulting embryos into female mice. Only 7 live pups resulted from 630 embryo transfers.

Over the years, Hayashi has built a reputation for taking on tasks that are both formidable and imaginative. “I admire his work,” says Azim Surani, a developmental biologist and former mentor of Hayashi’s at the University of Cambridge, UK. “It’s very original.”

Collaborators past and present also point to Hayashi’s patience and work ethic. “He works like ten people,” says Daisuke Kitamura, an immunologist at Tokyo University of Science who supervised Hayashi’s PhD research.

Hayashi brushes aside this praise, too: “I really struggle to get everything done,” he says. “Sometimes I can’t manage to finish my work on time.”

Hayashi has wanted to study germ cells — those that naturally give rise to sperm and egg cells — since his undergraduate days, drawn by both their importance for reproduction, and how they live on through subsequent generations. “The germ line is the source of life,” he says. “And it is eternal. It is the only cell line that can survive indefinitely after differentiation.”

Now, his lab is looking to extend the work in mice to another animal: the northern white rhinoceros (Ceratotherium simum cottoni). There are only two known northern white rhinos alive today, and both are female.

Hayashi’s techniques could be a way to save the lineage, but growing rhinoceros sperm and eggs in the lab is much more difficult than working with mice, he says.

The complexity of working with human reproductive cells would be greater still. It could be decades before the techniques that Hayashi has pioneered to create egg and sperm could be used in humans, says Amander Clark, a developmental and stem-cell biologist at the University of California, Los Angeles. In the meantime, Hayashi largely stays out of ethical discussions that such work poses. “From the viewpoint of the scientist, we are relatively simple: we try to produce oocytes that have good quality,” he says. “But whether these kinds of oocytes should be used is not our decision. This decision should be made by society.”

Nature 624, 499 (2023)

doi: https://doi.org/10.1038/d41586-023-03922-6

Reprints and permissions

NEWS FEATURE 13 December 2023

Nuclear-fusion breakthrough: this physicist helped to achieve the first-ever energy gain

Annie Kritcher and her team at the US National Ignition Facility designed fusion experiments that generated more energy than they consumed.

By

Jeff Tollefson

Credit: Cayce Clifford for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

Physicist Annie Kritcher rode a wave of optimism into 2023. Weeks earlier, she had helped the US Department of Energy’s National Ignition Facility (NIF) to achieve a goal that had eluded laboratories around the world for decades: compressing atoms so tightly that their nuclei fuse, and generating more energy than the reaction consumes.

But after reaching that experimental milestone, known as ignition, the pressure was on for a repeat performance.

Costing US$3.5 billion and housed at Lawrence Livermore National Laboratory in California, the NIF was designed to bolster nuclear-weapons science. Advances there could also help to develop nuclear fusion as a safe, clean and almost limitless source of energy. The NIF’s successful experiment last year came as a surprise to many. Ignition had been a decade behind schedule, and some feared that it was beyond reach. As the lead designer of the main fusion experiments, Kritcher and her team immediately set out to prove that the NIF could reliably achieve ignition.

High-stakes research rarely goes smoothly: the team’s first attempt at a repeat, in June, came up just shy. “It can get crazy, and I do get stressed,” says Kritcher.

The next shot paid off. On 30 July, the facility’s 192 laser beams delivered 2.05 megajoules of energy to a frozen pellet of the hydrogen isotopes deuterium and tritium, suspended in a gold cylinder. The resulting implosion caused the isotopes to release energy as they fused into helium, generating temperatures six times hotter than the core of the Sun. The reactions produced a record 3.88 megajoules of fusion energy.

Other facilities have generated more fusion energy over longer periods of time, most notably in tokamak reactors, which use powerful magnetic fields to confine fusion reactions. This is the technology under development by the $22-billion ITER project, an international collaboration near Saint-Paul-lez-Durance, France. Before the NIF’s achievement, however, no lab had produced a fusion reaction that generated more energy than it had consumed.

Kritcher and her team followed up their July success with two more ignition shots in October, bringing the total to four successful runs out of the last six shots. They are gearing up for even higher yields next year. In doing so, scientists at the facility have unlocked research opportunities and helped to fuel a wave of optimism about the future of fusion energy.

The NIF is designed to help government scientists to ensure the safety and reliability of the US arsenal of thermonuclear weapons without test detonations, but that’s not what drew Kritcher to the lab initially. She had worked on fusion energy during a summer internship at Livermore in 2004, before beginning her graduate studies, and quickly set her sights on the NIF as one of the only places on Earth to study fusion reactions.

She joined the NIF in 2012 and became a lead designer in 2016. Since then, she has led a team that analyses experimental data and uses computer models to design experiments aimed at achieving and increasing fusion yields, by tweaking parameters such as the size and configuration of the target and the energy and timing of various laser beams. Once her team finishes the design, the lab’s experimental team takes over to fire the laser and collect the data.

“Annie was an excellent student, fully dedicated to her work,” says Roger Falcone, a physicist at the University of California, Berkeley, who worked with Kritcher from when she was a graduate student through to her early days at the NIF. During that time, she demonstrated her strength in designing laser experiments to test how materials behave when compressed to extreme temperatures and pressures, he says.

Those skills pulled Kritcher into the centre of the fusion programme in 2016. The energy yields from fusion experiments had plateaued, and NIF chief scientist Omar Hurricane wanted to lay out a fresh path. Kritcher came forward with ideas, says Hurricane. “She jumped in with both feet,” he says, and that’s when she became one of the NIF’s lead designers.

Kritcher and her team spent the next several years running numbers and adjusting designs on what became the NIF’s main experimental effort. Alongside making a variety of changes to the target, they were able to capitalize on improvements that boosted the overall laser energy available. The result, with increasing regularity, is fusion.

With the formal ignition goal under her belt, Kritcher is already working on a fresh series of experiments that aims to boost the yield yet again by delivering even more laser energy to a thicker target capsule. This could represent another step towards the NIF’s goal of achieving yields of tens of megajoules and beyond.

In the long run, she is confident that the facility, with some upgrades, will be able to achieve its goals and increase the yields by an order of magnitude, which would put scientists in a position to begin work on a prototype laser fusion energy reactor. She says it’s not a question of if, but when, fusion energy will arrive, and she is hopeful that lasers will play a part.

“I think it’s a very good possibility,” she says, “and I would love to be involved.”

Nature 624, 500-501 (2023)

doi: https://doi.org/10.1038/d41586-023-03923-5

Reprints and permissions

NEWS FEATURE 13 December 2023

How Earth’s first global heat officer is tackling climate change

Eleni Myrivili is helping the world to prepare for the threats of climate change as the United Nations chief heat officer.

By

Alexandra Witze

Credit: Eirini Vourloumis/The New York Times/Redux/eyevine

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

While temperatures soared and wildfires raged through the forests of Greece this July, Eleni Myrivili watched with horror from her home in Athens. “This is the type of devastation that you can’t really replace,” she says. “And this is clearly a catastrophe that we are responsible for.”

Myrivili, a former deputy mayor of Athens, now has a global role working to mitigate the catastrophic impacts of the warming climate. She is the first United Nations chief heat officer — and has a remit to keep people cool as the planet boils.

This July broke unofficial records for the hottest average global temperatures on a single day and month, and 2023 will almost certainly be the hottest year on record by the end of December. That extreme heat is taking a deadly toll. As just one example, a paper published in July estimated that heatwaves in Europe in 2022 killed more than 61,000 people between the end of May and early September, with Italy, Greece and Spain seeing the highest heat-related mortality rates (J. Ballester et al. Nature Med. 29, 1857–1866; 2023).

Myrivili’s focus on heat is a long way from the start of her career as a cultural anthropologist. She got a PhD studying immigration, violence and borderlands, specializing in the region where Albania, North Macedonia and Greece meet. She then started teaching at the University of the Aegean in Μytilene, Greece. In summer 2007, parts of the country ignited in flames and much of Parnis National Park near Athens burned. “All of us who work in climate change have these moments where you kind of wake up to the reality of climate change,” she says. “For me, it was 2007.”

Angered by the lack of information and advice about the fires, Myrivili decided to go into politics. After a stint working with a Greek green political party, she read the 2013 book If Mayors Ruled the World by Benjamin R. Barber and decided to pivot to city government.

Taking various positions in Athens’s government, she worked to infuse climate resilience and planning across city departments. When she became head of the parks department, she changed her title to chief of urban nature, resilience and climate-change adaptation to underscore what she saw as her focus. In 2021, she was named chief heat officer for Athens in a wave of such appointments in various cities, organized by the Atlantic Council’s Adrienne Arsht–Rockefeller Foundation Resilience Center in Washington DC.

To raise awareness of deadly weather patterns, Myrivili kicked off the practice in Greece of naming heatwaves. She also worked to secure funding for climate initiatives; in 2018, her team landed a €5-million loan (US$5.9 million at the time) from the European Investment Bank for climate-adaptation projects, including planting green spaces in Athens.

Myrivili’s hard work and connections made her effective at gaining traction in city government, says Elissavet Bargianni, who succeeded Myrivili as the Athens chief heat officer earlier this year. “A lot of people have trust in her,” Bargianni says. “She can take something that may seem impossible and find a way to make it possible.”

Now, Myrivili is focused on raising awareness of extreme heat at the global level, and on securing money for projects through the auspices of UN Habitat, the programme that works towards making cities sustainable. That includes a global-cooling pledge introduced at the COP28 climate conference in Dubai this month. It aims to support the development and roll-out of cooling technologies that do not add to greenhouse-gas emissions.

Just don’t mention air conditioning to Myrivili, who hates it for its climate impacts. But during a heatwave in 2021, she finally bought a small air-conditioning unit for her bedroom. “I still hate it,” she says.

And yet the use of air conditioners is only going to grow as temperatures climb. That’s one of the reasons Myrivili was at the Dubai climate negotiations that are trying to stop the world from burning — and aiming to prepare for what’s to come.

Nature 624, 502 (2023)

doi: https://doi.org/10.1038/d41586-023-03924-4

Reprints and permissions

NEWS FEATURE 13 December 2023

OpenAI’s chief scientist helped to create ChatGPT — while worrying about AI safety

Ilya Sutskever has played a key part in developing the conversational AI systems that are starting to change society.

By

Nicola Jones

Credit: Jim Wilson/The New York Times/Redux/eyevine

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

As a teenager, Ilya Sutskever knocked on Geoffrey Hinton’s door at the University of Toronto in Canada and asked for a job. “He said he was cooking fries to make money over the summer, and he would rather be working for me doing AI,” says Hinton, who is often recognized as the godfather of modern artificial intelligence (AI).

Hinton gave the ambitious student some papers to read, and Sutskever came back wondering why the authors hadn’t worked out what seemed, to him, obvious solutions. “All his instincts were correct,” says Hinton. He credits Sutskever with being a visionary pioneer in deep learning and large language models (LLMs) — the roots of conversational AI bots, such as ChatGPT, that have captivated the world this year. “It’s not just intelligence” that sets him apart, says Hinton. “It’s the urgency with which he gets on with things.”

Sutskever became the chief scientist at OpenAI in San Francisco, California, where he has had a central role in developing ChatGPT. But he’s also worried about the future of AI. This July, he shifted focus to co-lead OpenAI’s four-year ‘superalignment’ project, which the company said will use 20% of its computing power to study how “to steer and control AI systems much smarter than us”.

A tension between safety and the commercial incentive to move fast might have contributed to a bewildering drama this November, in which Sutskever played a key part in firing and then rehiring OpenAI’s chief executive, Sam Altman. After the upheaval, Sutskever declined to talk to Nature.

To some, Sutskever’s vision is admirable. “He’s got a very strong moral compass,” says Hinton. “He really is very concerned about AI safety.” But others say that focusing on how to control yet-to-arrive AI systems distracts from the real and present dangers of the technology. It “puts intervention way off on the horizon”, says Sarah Myers West, managing director of the AI Now Institute, a policy-research organization in New York City. Instead, she says, we need “to tackle near-term harms”, such as AI systems reinforcing biases in their training data, or potentially leaking private information.

The lack of transparency about AI systems is also a concern. OpenAI and some other companies keep their code and training data private. Sutskever says that in the long term, closed systems will be the responsible course to avoid letting others make powerful AIs. “At some point, the capability will become so vast that it will be obviously irresponsible to open-source models,” he said this April.

Sutskever, who was born in the Soviet Union in 1986, has always been a precocious learner, taking university-level classes in coding as a teenager in Israel. After his family moved to Canada, he began working with Hinton on deep learning in 2003. In 2012, Sutskever and another of Hinton’s students built AlexNet, a neural network that won a landmark image-recognition competition by a startling margin. Sutskever later moved to Google, where he helped to develop AlphaGo, which beat human champions at the complex board game Go.

In 2015, Sutskever was invited to dinner with Altman and others, including billionaire entrepreneur Elon Musk. They co-founded OpenAI as a non-profit organization that year, to “benefit humanity”. Sutskever saw it as a chance to take seriously the pursuit of artificial general intelligence (AGI), a system as smart as any person. “Researchers are somehow, I would say, trained to think small … But at OpenAI we took the liberty to look at the big picture,” he said earlier this year.

Wojciech Zaremba, another OpenAI co-founder, credits Sutskever with pushing the company to pour more effort into its Generative Pre-trained Transformer (GPT) system after the first incarnation of GPT-1 in 2018. Unlike many others at the time, Sutskever was convinced that scaling up computing power alone would make these systems smarter. “He understood that before almost anybody else,” says Hinton.

To attract the funding needed for more computing power, the team moved OpenAI from a non-profit to a ‘capped-profit’ model in 2019, enticing technology giant Microsoft to pour billions in cash and computing resources into the operation. That paid off: the LLM improved and ChatGPT, released in November 2022, became a sensation.

Amid this success, internal chaos erupted when Sutskever and other OpenAI board members fired Altman on 17 November this year. Many employees threatened to decamp to Microsoft with Altman — including Sutskever, who said he regretted his actions. He was removed from the board when Altman rejoined the company five days later.

Throughout, Sutskever has been consistently bold in his statements about AI. In 2022, he declared that AI might already be “slightly conscious”, variously evoking awe, terror and laughter. He says publicly that AGI and even ‘superintelligence’ surpassing the combined intellect of humanity could be developed within years or decades. “To this day, I’m surprised at how bullish he is,” says AI researcher Andrew Ng, for whom Sutskever worked as a postdoc at Stanford University in California in 2012.

But, adds Ng, Sutskever “has the admirable trait of being able to pick a direction and pursue it relentlessly, regardless of whether others agree with him or not”.

Nature 624, 503 (2023)

doi: https://doi.org/10.1038/d41586-023-03925-3

Reprints and permissions

NEWS FEATURE 13 December 2023

Superconductivity debunker: this physicist exposed flaws in a blockbuster claim

James Hamlin found problems with the work of controversial physicist Ranga Dias.

By

Dan Garisto

Credit: Zach Stovall for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

James Hamlin remembers the first time an experiment deceived him. As a graduate student, Hamlin saw signs of superconductivity — electrons flowing without resistance — in an unexpected material. Excited, Hamlin shared the news with his adviser, who was unfazed. “He asked me lots of questions and suggested lots of additional measurements,” Hamlin says. On further inspection, the superconductivity signal disappeared. The lesson he imbibed was straightforward: “Don’t assume you’ve discovered something,” Hamlin says.

That lesson played out on an international stage this year when Ranga Dias, a physicist at the University of Rochester, New York, reported in Nature in March that he had achieved the long-sought goal of room-temperature superconductivity, in a material held under moderate pressure.

Amid a furore of hype and criticism, Hamlin, a physicist who conducts high-pressure experiments at the University of Florida in Gainesville, and Brad Ramshaw, a superconductivity researcher at Cornell University in Ithaca, New York, sent Nature their concerns about the research. (Nature’s news team is independent from its journals team.) The paper was retracted in November, generating headlines: it was Dias’s third retraction in little more than a year.

This wasn’t the first or even the second time that Hamlin had exposed problems with Dias’s work. In 2020, Dias had published a paper in Nature that also claimed to have discovered the first room-temperature superconductor, albeit at much higher pressure. All known superconductors must be kept either extremely cold or at high pressures to function. One that works at ambient temperature and pressure might permit applications such as magnets for magnetic resonance imaging (MRI) that wouldn’t need expensive cooling equipment, and highly efficient computer chips — tantalizing possibilities that have led to hype around speculative claims of room-temperature superconductivity.

After Dias’s 2020 paper came out, Jorge Hirsch, a physicist at the University of California, San Diego, thought a measurement in that study looked iffy — and had similarities to a measurement in a 2009 paper that Hamlin had co-authored. Pushed by Hirsch, Hamlin investigated his own work and found evidence that another co-author, Matthew Debessai, had manipulated those data. (Debessai, who no longer works in research, didn’t respond to a request for comment.) That paper was retracted in 2021, but Hamlin wondered whether there were problems with Dias’s 2020 study as well.

It took more than a year for Dias and a co-author, Ashkan Salamat, a physicist at the University of Nevada, Las Vegas, to post the data Hirsch wanted. Analyses by Hirsch, Hamlin and others found evidence of manipulation. In September 2022, Nature retracted the work; the retraction statement did not mention misconduct, and Dias denied wrongdoing.

Hamlin also found that his own and others’ work had been plagiarized in Dias’s thesis — and that Dias had reused some of the thesis data in a later paper in Physical Review Letters. That paper was retracted this August. (Dias disagreed with the retraction, although he acknowledged not providing “explicit attribution” for some of his thesis.)

When Hamlin laid out his evidence at a virtual workshop this March, some observers were “stunned” at the work he’d put in, says Brian Skinner, a physicist at Ohio State University in Columbus and an organizer of the conference. At one point, unable to access raw electrical-resistance data, Hamlin created a tool to extract data directly from Dias’s graphs. “It was pretty heroic,” Skinner says.

This background of controversy was why many researchers were surprised that Nature published a second Dias paper in March, with another room-temperature superconductor claim, albeit in a different material.

This time, a lot of the raw data was public and questions quickly emerged online. Ramshaw and Hamlin focused on a few central concerns, including whether Dias had actually measured the electrical resistance going to zero.

In a subsequent back-and-forth involving Ramshaw, Hamlin and editors at Nature, Dias and his co-author Salamat did not explain how they had obtained this measurement. “We couldn’t get a square, straight answer on this very simple question,” Hamlin says. Nature’s news team reached out to Dias and Salamat for comment but did not receive any reply.

Then, in September, 8 of the paper’s 11 co-authors (including Salamat but not Dias) requested a retraction, corroborating concerns raised by Hamlin and Ramshaw. Nature retracted the paper on 7 November, citing concerns about data integrity.

Hamlin and Ramshaw say data availability made the latest retraction easier: it took only half a year, rather than two years.

Shanti Deemyad, a high-pressure experimentalist now at the University of Utah in Salt Lake City, supervised Hamlin in the laboratory when he was an undergraduate. She’s not surprised by his dedication. “He was very ambitious and very excited,” she says. “And he wanted to know all the details.” Even when she showed up to the lab as early as 6.30 a.m., Hamlin was there too, eager to learn.

Hamlin isn’t a full-time sleuth and is keen to spend more time on his own superconductivity research. “It’s still really the topic in physics that I find most exciting,” he says. “The BS of human beings is much less interesting to discover than the mysteries of nature.”

Nature 624, 504-505 (2023)

doi: https://doi.org/10.1038/d41586-023-03926-2

Reprints and permissions

NEWS FEATURE 13 December 2023

Weight-loss-drug pioneer: this biochemist finally gained recognition for her work

Svetlana Mojsov led early studies of GLP-1, the hormone behind Wegovy, Ozempic and other blockbusters.

By

Elie Dolgin

Credit: Chris Taggart courtesy of The Rockefeller University

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

A new class of weight-loss drugs has swept into clinics and made medicines such as Ozempic and Wegovy household names. They have generated billions of dollars in profit for the pharmaceutical industry and brought scientific acclaim for the researchers credited with discovering the hormone behind them: an appetite suppressant called glucagon-like peptide-1 (GLP-1).

But there is one early pioneer who has not received due acknowledgement: Svetlana Mojsov. A biochemist now at The Rockefeller University in New York City, Mojsov had a pivotal role in identifying and characterizing the active form of GLP-1. Yet her efforts went unrecognized in many accounts of the hormone’s discovery, and she has not shared the scientific prizes bestowed for that feat.

This year, Mojsov fought the entrenched narratives — and began to win wider recognition for her contributions to the field. “All I’m trying to do is put the scientific record straight,” she says.

A Yugoslavian-born scientist now in her mid-seventies, Mojsov was a member of the endocrine unit at Massachusetts General Hospital (MGH) in Boston in the 1980s, where she also directed a facility that made synthetic proteins for use in the unit and beyond. During this time, she conducted a series of landmark studies and supplied others with the research tools needed to make their own advances.

Her own work with GLP-1 started when she predicted that a particular version of the hormone should exist in mammalian gut tissue. Mojsov then experimentally confirmed that prediction (S. Mojsov et al. J. Biol. Chem. 261, 11880–11889; 1986). She next showed that this biologically active form of GLP-1 could trigger insulin release from the pancreas of a rat (S. Mojsov et al. J. Clin. Invest. 79, 616–619; 1987).

Peptides and antibodies created by Mojsov were also instrumental to several other GLP-1 experiments performed in cell lines at the time, and allowed clinicians to demonstrate that GLP-1 could lower blood glucose in an early human trial.

This research set the stage for drugs such as Ozempic and Wegovy, both of which feature a GLP-1 analogue called semaglutide. It incorporates only minor modifications from the peptide outlined in Mojsov’s original paper; the changes improve stability and ensure longer-lasting effects. Global sales of semaglutide are now worth more than US$1 billion a month, and this class of drugs is forecast to become one of the best-selling medicines of all time.

Yet, Mojsov’s part in the discovery was long overlooked.

She had to endure a protracted legal battle to have her name added to foundational patents as a co-inventor, a move that earned Mojsov royalties for a year or two connected to sales of a first-generation GLP-1 drug. But, with her patents long expired, she has no financial stake in the semaglutide windfall.

The lack of recognition started to grate on Mojsov. History was being “manipulated”, she says. For example, she felt that commentaries published to coincide with prizes exaggerated some of the winners’ contributions at her expense.

“That was a wake-up call for me,” Mojsov says. She began to speak out.

At her urging, journals such as Cell and Nature have revised narratives of GLP-1’s discovery to better reflect Mojsov’s involvement at MGH — one of the two places, along with the University of Copenhagen, where researchers independently homed in on and characterized the active form of the hormone. In September 2023, the journal Science and the news outlet STAT published two lengthy profiles that, some 40 years after she began working on GLP-1, finally told her side of the story.

E-mails of support have since poured in — from fellow scientists, especially women, who felt that their own work had been sidelined, and from others frustrated by the hierarchies of biomedical research.

The attention focused on Mojsov has also begun to change the minds of some who were involved in early GLP-1 research.

“She’s got a point,” says Joel Habener, a molecular endocrinologist at MGH. Habener collaborated with Mojsov, appearing as the senior author on all her seminal papers, but was the sole patent holder before Mojsov had the patents corrected. “She absolutely deserves to be recognized,” he says.

In past accounts of the discovery, Mojsov was mischaracterized as a scientist in Habener’s group, rather than an independent investigator whose efforts helped to propel the MGH endeavour. “Her contributions were essential,” says Richard Goodman, a molecular neurobiologist at Oregon Health & Science University in Portland who, as a postdoc of Habener’s, helped to decode the gene behind a precursor to GLP-1. “Would it have moved forward without Svetlana? No.”

Prizes — and the prestige they bring — could follow, but that is not the top priority for Mojsov, who continues to study GLP-1 and related proteins in her lab.

“I am just happy that my work is recognized,” she says. “Everything else is secondary.”

Nature 624, 506 (2023)

doi: https://doi.org/10.1038/d41586-023-03927-1

Reprints and permissions

NEWS FEATURE 13 December 2023

Malaria fighter: this researcher paved the way for a game-changing vaccine

Halidou Tinto runs a clinic in rural Burkina Faso that has been instrumental to the approval of the world’s first malaria vaccines.

By

Brendan Maher

Credit: Kilayé Bationo for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

In October, work and life collided for Halidou Tinto when his six-year-old daughter caught malaria. A director of clinical trials for malaria drugs and vaccines for more than a decade, Tinto knew how severe the disease could be. His daughter was hospitalized for four days with a fever, headaches and vomiting. She recovered, but “it was really serious”, he says.

That same month, a vaccine called R21 that he had been testing was recommended for use by the World Health Organization (WHO). It is only the second malaria vaccine to be approved and many think it could prevent millions of deaths in Africa, where the vast majority of malaria infections occur. Every year there are more than 200 million cases and 500,000 deaths on the continent, predominantly in children younger than 5 years old.

The institute that Tinto directs, the Clinical Research Unit of Nanoro (CRUN) in Burkina Faso, is a key test site for R21, its predecessor RTS,S and several other drugs. Many scientists credit Tinto’s diligence for the institute’s success.

Tinto earned a PhD at the University of Antwerp in Belgium, studying how malaria becomes resistant to various drugs. His adviser at the time, Umberto D’Alessandro, a clinical epidemiologist now at the London School of Hygiene & Tropical Medicine and based in Fajara, the Gambia, says that he has always been struck by both Tinto’s rigour as a scientist and his dedication. “He truly wants to advance science and research in Africa,” says D’Alessandro.

Tinto had an opportunity to do a postdoc at a US university, but turned it down to return to Burkina Faso in 2006. There, he helped to establish the CRUN with local scientists and clinicians.

In 2007, pharmaceutical company GSK and its partners were gearing up to do late-stage clinical trials of RTS,S — a vaccine that had been in development for years. For Tinto’s new clinic, with just ten employees, becoming part of the trial seemed like a long shot. “They were surprised that we applied,” he says, “because there was no electricity, no cars, nothing.” Nevertheless, Tinto convinced the coordinators that he could make it work.

He met with the king of the village, and together they persuaded the Burkina Faso government to connect Nanoro to the national grid. The CRUN produced data that helped to get RTS,S approved in Africa.

That vaccine has been associated with a significant reduction in child mortality. But GSK can produce only a few million doses a year. Even if Burkina Faso got one million of those, Tinto says, that would vaccinate only 250,000 children a year. “We still have millions of children lagging behind,” he says. That’s why people are excited about R21: the Serum Institute of India in Pune can currently produce 100 million doses a year. R21 should also be more affordable than RTS,S, and some researchers expect it to be more effective.

Tinto ran an influential early study of the vaccine, starting in 2019 (M. S. Datoo et al. Lancet 397, 1809–1818; 2021). “He led the trial that really, to people in the field, showed that this vaccine was going to be different,” says Adrian Hill, a vaccinologist at the University of Oxford, UK, who oversaw the development of R21.

The WHO has said that R21 will be available across Africa as early as mid-2024. Meanwhile, Tinto is working on more than 30 clinical trials, including two further malaria vaccines and more studies on R21.

CRUN has expanded beyond Nanoro, and now has more than 400 staff members and associates, including dozens of graduate students from all over Africa. D’Alessandro, with whom Tinto still collaborates, says that it is a good example of how research can stimulate development in Africa. But what inspires Tinto the most is the opportunity to save lives. “You cannot have really any other satisfaction beyond that; because life, for me, is the most important thing.”

Nature 624, 507 (2023)

doi: https://doi.org/10.1038/d41586-023-03928-0

Reprints and permissions

NEWS FEATURE 13 December 2023 Correction 15 December 2023

The cancer physician who helped to deliver a life-extending treatment

Thomas Powles’s breakthrough success in treating a deadly bladder cancer could herald the next wave of powerful immunotherapeutic drugs.

By

Carissa Wong

Credit: Anna Lukala for Nature

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

The results looked almost too good to be true as Thomas Powles combed through data from a clinical trial for advanced bladder cancer. “I thought, ‘oh we’ll find something wrong somewhere’,” says Powles, a cancer researcher at St Bartholomew’s Hospital and Queen Mary University of London. But he never did.

Compared with standard chemotherapy, a combination of two newer drugs seemed to extend median survival time from about 16 months to 2.5 years. In October, when he presented the data at the European Society for Medical Oncology Congress in Madrid, the audience erupted into applause — twice. “It was very emotional,” says Powles.

“This particular trial is absolutely the biggest breakthrough in the treatment of advanced bladder cancer that we’ve had in roughly 40 years,” says Eila Skinner, a bladder-cancer researcher at Stanford University in California. The drug cocktail is the first to perform better than the standard treatment that’s been in use since the 1980s.

The exciting results add to a growing number of studies that highlight the promise of a recently developed class of treatments called antibody–drug conjugates (ADCs), says cancer researcher Funda Meric-Bernstam at the University of Texas MD Anderson Cancer Centre in Houston. ADCs consist of a toxic chemotherapeutic drug bound to an antibody that specifically targets tumour cells. Promising results from studies on other types of cancer have also shown that ADCs can slow tumour growth and extend the survival of those treated (C. Dumontet et al. Nature Rev. Drug Discov. 22, 641–661; 2023). “I think ADCs are the next pillar of cancer care,” says Meric-Bernstam. “It’s clear that we’ll be able to transform the outcomes of several tumour types.”

The bladder-cancer trial tested an ADC called enfortumab vedotin, which targets a protein, called nectin-4, that is abundant on bladder cancer cells. It was partnered with another immunotherapy drug, pembrolizumab, which takes the brakes off of cancer-killing immune cells.

This one–two punch was approved by the US Food and Drug Administration earlier this year for people who are ineligible for a type of chemotherapy called cisplatin — which includes roughly half of all people with advanced bladder cancer.

Approval for wider use of the drug could happen early next year, says bladder-cancer researcher Guru Sonpavde at the Advent Health Cancer Institute in Orlando, Florida.

Powles nearly became a cardiologist after his medical training. But he saw that there could be greater opportunities for discovery in the area of bladder cancer. “You’ve had the same treatment for, what was then, 20 years. Life expectancy is 12 months, and unfortunately almost everybody dies of the disease,” he says. “A very, very small part of this map has been charted.”

He has led more than 20 randomized clinical trials, including several that focused on applying advanced immunotherapeutic drugs to kidney and bladder cancer. Many of the trials failed to show improved benefit, but Powles has persevered, convinced that there was a way to boost survival rates. The biggest challenge is persuading research funders to invest. “Sometimes we get caught in a storm on that journey to map out those blank areas, but even with the negative trials, we’re finding new stuff,” says Powles.

Sonpavde, who has known Powles for more than a decade, says that Powles comes up with great ideas and is able to follow them through. “A lot of people have ideas,” he says, but those ideas “don’t get executed”.

Powles is a fan of travelling and paints in his spare time. He has been developing a series of aerial cityscapes, which play on his affinity for mapping out complex systems. “I can get lost in that. If I’m having a frustrating day, I’ll just go and adjust one of them,” he says.

But besides his desire for discovery, Powles says he is driven by the people he treats. “Patients who were involved in these trials are really the people who are making an extraordinary sacrifice,” he adds.

Nature 624, 508 (2023)

doi: https://doi.org/10.1038/d41586-023-03929-z

Updates & Corrections

- Correction 15 December 2023: The affiliation for Thomas Powles omitted Queen Mary University of London.

Reprints and permissions

NEWS FEATURE 13 December 2023

ChatGPT and science: the AI system was a force in 2023 — for good and bad

The poster child for generative AI software is a startling human mimic. It represents a potential new era in research, but brings risks.

By

Richard Van Noorden &

Richard Webb

Credit: Olga Yastremska/Alamy, Gabby Jones/Bloomberg via Getty

This story is part of Nature’s 10, an annual list compiled by Nature’s editors exploring key developments in science and the individuals who contributed to them.

It co-wrote scientific papers — sometimes surreptitiously. It drafted outlines for presentations, grant proposals and classes, churned out computer code, and served as a sounding board for research ideas. It also invented references, made up facts and regurgitated hate speech. Most of all, it captured people’s imaginations: by turns obedient, engaging, entertaining, even terrifying, ChatGPT took on whatever role its interlocutors desired — and some they didn’t.

Why include a computer program in a list of people who have shaped science in 2023? ChatGPT is not a person. Yet in many ways, this program has had a profound and wide-ranging effect on science in the past year.

ChatGPT’s sole objective is to plausibly continue dialogues in the style of its training data. But in doing so, it and other generative artificial-intelligence (AI) programs are changing how scientists work (see go.nature.com/413hjnp). They have also rekindled debates about the limits of AI, the nature of human intelligence and how best to regulate the interaction between the two. That’s why this year’s Nature’s 10 has a non-human addition.

Some scientists have long been aware of the potential of large language models (LLMs). But for many, it was ChatGPT’s release as a free-to-use dialogue agent in November 2022 that quickly revealed this technology’s power and pitfalls. The program was created by researchers at OpenAI in San Francisco, California; among them was Ilya Sutskever, also profiled in this year’s Nature’s 10. It is built on a neural network with hundreds of billions of parameters, which was trained, at a cost estimated at tens of millions of dollars, on a giant online corpus of books and documents. Large teams of workers were also hired to edit or rate its responses, further shaping the bot’s output. This year, OpenAI upgraded ChatGPT’s underlying LLM and connected it to other programs so that the tool can take in and create images, and can use mathematical and coding software for help. Other firms have rushed out competitors.

For some researchers, these apps have already become invaluable lab assistants — helping to summarize or write manuscripts, polish applications and write code (see Nature 621, 672–675; 2023). ChatGPT and related software can help to brainstorm ideas, enhance scientific search engines and identify research gaps in the literature, says Marinka Zitnik, who works on AI for medical research at Harvard Medical School in Boston, Massachusetts. Models trained in similar ways on scientific data could help to build AI systems that can guide research, perhaps by designing new molecules or simulating cell behaviour, Zitnik adds.

But the technology is also dangerous. Automated conversational agents can aid cheats and plagiarists; left unchecked, they could irreversibly foul the well of scientific knowledge. Undisclosed AI-made content has begun to percolate through the Internet and some scientists have admitted using ChatGPT to generate articles without declaring it.

Then there are the problems of error and bias, which are baked into how generative AI works. LLMs build up a model of the world by mapping language’s interconnections, and then spit back plausible samplings of this distribution with no concept of evaluating truth or falsehood. This leads to the programs reproducing historical prejudices or inaccuracies in their training data, and making up information, including non-existent scientific references (see W. H. Walters & E. I. Wilder Sci. Rep. 13, 14045; 2023).

Emily Bender, a computational linguist at the University of Washington, Seattle, sees few appropriate ways to use what she terms synthetic text-extruding machines. ChatGPT has a large environmental impact, problematic biases and can mislead its users into thinking that its output comes from a person, she says. On top of that, OpenAI is being sued for stealing data and has been accused of exploitative labour practices (by hiring freelancers at low wages).

The size and complexity of LLMs means that they are intrinsically ‘black boxes’, but understanding why they produce what they do is harder when their code and training materials aren’t public, as in ChatGPT’s case. The open-source LLM movement is growing, but so far, these models are less capable than the large proprietary programs.

Some countries are developing national AI-research resources to enable scientists outside large companies to build and study big generative AIs (see Nature 623, 229–230; 2023). But it remains unclear how far regulation will compel LLM developers to disclose proprietary information or build in safety features.

No one knows how much more there is to squeeze out of ChatGPT-like systems. Their capabilities might yet be limited by the availability of computing power or new training data. But the generative AI revolution has started. And there’s no turning back.

Nature 624, 509 (2023)

doi: https://doi.org/10.1038/d41586-023-03930-6

Reprints and permissions

Books & Arts

- Black holes, love and poetry — an artistic exploration of intimacy and adventure

- Prebunking propaganda, and how to live well: 2023’s best of Books in brief

- Does quantum theory imply the entire Universe is preordained?

BOOK REVIEW 19 December 2023

Black holes, love and poetry — an artistic exploration of intimacy and adventure

A book by physicist Kip Thorne and artist Lia Halloran explores the mysteries of space through poetry and paintings.

By

Ron Cowen

Warped space-time around a black hole, as portrayed by artist Lia Halloran.Credit: Lia Halloran

The Warped Side of Our Universe: An Odyssey through Black Holes, Wormholes, Time Travel, and Gravitational Waves Kip Thorne & Lia Halloran Liveright (2023)

Physicist Kip Thorne and visual artist Lia Halloran began to collaborate on a magazine article about the strange, warped space-time in and around a black hole more than a decade ago. It was never published — but it inspired a much more ambitious project.

The pair have just released an illustrated book portraying space-time storms generated by colliding black holes and neutron stars, as well as wormholes and the possibility of time machines — with explanations and illustrations all guided by cutting-edge computer simulations. It’s an intimate account, too. Halloran’s paintings depict her wife, Felicia, with her body stretching, spinning and contorting as she nears the gravitational maw of a black hole. Thorne expresses his words in verse.

Thorne, who is professor emeritus at the California Institute of Technology in Pasadena, won a share of the 2017 Nobel Prize in Physics for his contributions to the development of the Laser Interferometer Gravitational-Wave Observatory (LIGO), the facility that made the first direct detection of the ripples in space-time known as gravitational waves. Halloran is chair of the art department at Chapman University in Orange, California. They spoke to Nature about their book, The Warped Side of Our Universe.

You say you’ve learnt a lot from each other. What, for example?

Thorne: Yes, the structures of singularities — locations of infinite density — inside a black hole. Singularities have always been drawn basically as points, but that does not begin to capture what the mathematics tells us. Physical laws say that as you approach a singularity, things get alternately stretched and squeezed in a manner that is truly chaotic. How do you depict that?

Halloran: I painted what Kip was describing about singularities: something that has geometrical shapes and surprises and chaos. But when I finished and looked at the painting, I also understood more clearly what he was saying — and that was a surprise. Singularities have perplexed me as an artist, and fascinated me. Terms such as ‘infinitely dense’ and ‘infinitely massive’ boggle my mind. How we understand the world we’re in drives my studio practice. I have taken physics and astronomy classes. I never wanted to be a scientist, but I loved the way science challenged me.

Lia Halloran (left) and Kip Thorne have been collaborating for more than a decade.Credit: Adam Ottke/Christopher Michel

Kip, why did you write in verse?

Thorne: Verse forces the mind to focus. If I’m writing in prose, there’s lots of detail and the focus is not nearly so intense. The verse sucks out the essence of what’s going on and conveys that in a manner that is almost visceral. And that influences how the reader thinks about it, and how I think about it.

When I was a boy in Logan, Utah, I had a newspaper delivery route and to alleviate boredom I would memorize poetry. What stuck with me most was the poetry of Robert Service — in particular, the ballad ‘The Cremation of Sam McGee’ — the humour in it and the alternating meter. I remembered the poem all these years later, and it’s the meter that I use in the book.

Isn’t that quite a departure for you?

Thorne: Early in my career, I co-wrote the textbook Gravitation (1973) with Charles Misner and John Wheeler. That was a landmark in terms of introducing new elements into a scientific textbook, filled with boxes, marginal comments, different fonts and a long list of other tools. I took great pride in introducing a new style of communicating science.

I feel that in my collaboration with Lia, I’ve accomplished that for the second time: a book that is a new way of communicating science to non-scientists. The integrated painting and the verse — it’s a very different style in a very different vehicle that conveys a very different aspect of science than anything that I’ve ever done before or that I’ve seen before.

Kip, you’ve said that when someone falls into a black hole, their head will twist one way and their feet the other. How do you know?

Thorne: It’s obvious in the mathematics, but computer simulations rub your nose in things that you had not noticed. They capture the twisting much better. We could not have seen this in the simulations of 15 years ago; that was far beyond our capabilities.

And, Lia, why depict your wife in peril near a black hole?

Halloran: We were not focused on the peril, more on the adventure of getting close to a black hole. As well as Felicia, the book depicts people who have been integral to Kip’s career. When have you seen a book that’s about a black hole and gravity that subtly talks about intimacy, about love, about friendship? We believe deeply that if the viewer can feel a personal connection, they are going to be even more curious and engaged in the material.

Thorne: Also, we found that as well as the violent singularity, there are also two gentle singularities inside the hole, which Felicia could survive. We don’t describe it, but Felicia might make it through those two singularities intact.

Kip, it took a team of 1,000 to build LIGO and you’ve told the Nobel Prize Committee that it should have given the prize to the team and not three people. Should the awards change to allow more recipients?

Thorne: I don’t see how the committee can avoid it much longer. It’s basically turning its backs on the nature of big science. That’s not by any means all science, but in physics and in chemistry and in biology, some pieces of science can be done only by large collaborations. And the power of large collaborations is something that the general public needs to understand and appreciate.

What are the next challenges for understanding gravity?

Thorne: Well, I think developing a theory of quantum gravity is the holy grail in theoretical physics at this time, because it controls the birth of the Universe and lots of other things — such as whether or not time machines would self-destruct when you try to turn them on, and what goes on inside a black hole.

Beyond that, there’s also the issue of how warped space-time behaves in the real Universe, and what kinds of phenomenon it produces — such as space-time storms when black holes collide and stretching and squeezing and twisting as you approach a chaotic singularity. With the discovery of gravitational waves, we have made our first contact with the warped side of the Universe, and there’s much more to learn about it.

And what do you think will happen in gravitational-wave research over the next decade?

Thorne: This year, the European Pulsar Timing Array and other observatories reported detecting a background of gravitational waves from colliding supermassive black holes, and perhaps from the birth of the Universe. Future discoveries, with LIGO and its successors on the ground as well as gravitational observatories in space, will deepen our understanding of warped space-time. Today, we’re in the same situation we were in four centuries ago, when Galileo built the first optical telescope. He and other astronomers discovered a new world—the richness of the Solar System. Now, we’re poised to discover the richness of the cosmos.

Nature 624, 511-512 (2023)

doi: https://doi.org/10.1038/d41586-023-04023-0

This interview has been edited for length and clarity.

Competing Interests

The author declares no competing interests.

BOOK REVIEW 18 December 2023

Prebunking propaganda, and how to live well: 2023’s best of Books in brief

Connoisseur Andrew Robinson gives ten of his favourite tomes from the year a more expansive review.

By

Andrew Robinson

Air bubbles in the ocean affect weather and climate all over the world.Credit: Getty

Blue Machine Helen Czerski Torva (2023)

Few scientific subjects are as vast as the ocean. Yet oceans “often seem invisible”, remarks physicist Helen Czerski.

Growing up in Manchester, UK, Czerski had access only to the freezing North Sea to the east and the grey Irish Sea to the west, neither of which much appealed. But why did her three physics degrees omit the ocean, she wonders?

Having finished a PhD in experimental explosive physics and looking for a new subject as a postdoc, she encountered a giant frame in her Californian laboratory that had buoys on the corners and waterproof boxes of sensors in the middle, used by colleagues to make marine measurements. “It was their gateway to another world”, she says, and it soon hooked her, too. She began research on the bubbles created by breaking ocean waves, and their influence on weather and climate across the world.

Czerski’s profound, sparkling book is a global ocean voyage mingling history and culture, animals and people, natural history and geography, in a quest to understand the physics of the “blue machine”: the ocean engine, powered by sunlight, that shunts energy from Equator to poles. “It has components on every scale, from the mighty Gulf Stream gliding across the Atlantic to the tiny bubbles bursting at the top of a breaking wave.”

The Good Life Robert Waldinger & Marc Schulz Simon & Schuster (2023)

Isolation imposed by the COVID‑19 pandemic highlighted the link between good relationships and happiness. Scientific evidence for the importance of relationships motivates this engrossing, moving examination, which is grounded in the unique Harvard Study of Adult Development. Established in 1938 to discover attitudinal predictors of healthy ageing, the project has followed the lives of three generations of the same families “from childhood troubles, to first loves, to final days”.

It includes more than 1,300 descendants of its original 724 participants (among them the then-future US president John F. Kennedy), making it “the longest in-depth longitudinal study of human life ever done”, according to its current director, psychiatrist Robert Waldinger, and associate director, psychologist Marc Schulz. Their book shows how the ‘good life’ — however defined — is never static, but unfolds as individuals keep striving for it. Nor is it easy. It inevitably includes “turmoil, calm, lightness, burdens, struggles, achievements, setbacks, leaps forward, and terrible falls”.

The happiest participants have been those, such as Kennedy, who managed to turn the question “What can I do for myself?” into “What can I do for the world beyond me?”, by deliberately creating good relationships with others.

The AI Dilemma Juliette Powell & Art Kleiner Berrett-Koehler (2023)

The benefits and harms of social media are intimately tied to the ongoing debate about artificial intelligence (AI). Will AI systems — trained partly on social media — benefit humanity or harm it?

Entrepreneur and technologist Juliette Powell and writer and educator Art Kleiner begin their exploration of this dilemma with the Moral Machine, a hugely popular online platform created in 2016 by the Media Lab at the Massachusetts Institute of Technology in Cambridge. The platform invited users around the world to test their attitudes by rapidly responding to dilemmas involving self-driving cars operated by AI, in which some passengers, pedestrians and pets must live while others must die. Unsurprisingly, most participants wanted to spare as many lives as possible, and favoured children over adults and humans over pets. Otherwise, their answers revealed no consensus. This “shows that people — especially those in different countries, cultures, and contexts — don’t agree on what we want”, comment the authors. “How, then, can we expect AI to know what priorities to control for?”

In their sometimes alarming analysis, the authors compare humans developing AI systems to first-time parents discovering the challenges of parenthood. They recommend that scientists and governments guide AI systems “as we would a child towards full adulthood”.

The World of Sugar Ulbe Bosma Belknap/Harvard Univ. Press (2023)

Sugar’s societal dominance is surprisingly recent. Granular sugar, manufactured by grinding sugarcane and heating the juice, is mentioned from as long ago as the sixth century bc in India, under the Sanskrit name śarkarā (the origin of French sucre and English ‘sugar’). But refined sugar became widely available in Europe only in the nineteenth century — boosted by mechanical means of production, but also by the transatlantic slave trade. Of the 12.5 million Africans kidnapped, up to two-thirds were enslaved on sugar plantations in the Americas, where labour conditions were much more lethal than on tobacco and coffee plantations.

Today, the average annual consumption of sugar and sweeteners per person is 40 kilograms in Western Europe. If the whole world were to match this level, global production would rise from 180 million tonnes per year to 308 million, entailing “an almost-proportionate gobbling up of land”.

The history of sugar is both a story of agricultural and economic progress and a bittersweet tale of “human exploitation, racism, obesity, and environmental destruction”, writes historian Ulbe Bosma in his authoritative, readable study — the first to be truly global. As he warns the reader, “We have not yet learned how to control it and bring it back to what it once was: a sweet luxury.”

People enslaved on sugar plantations endured treacherous and often deadly conditions.Credit: Heritage Art/Heritage Images/Getty

Foolproof Sander van der Linden 4th Estate/W. W. Norton (2023)

In April 2020, Michael Whitty — volunteer for a local charity, operator of an airport parking facility and father of three — set fire to a phone mast near his UK home. At least 50 UK masts have been burnt by others.

Police evidence from Whitty’s phone suggested that information he found online had convinced him that radiation from 5G masts was damaging people’s immune systems, and thus helping to spread the virus SARS-CoV-2. At least 10% of UK inhabitants accepted this debunked conspiracy theory, according to a 2020 survey by social psychologist Sander van der Linden, who first became fascinated by misinformation when he learnt about the Nazis’ propaganda, and their execution of many of his Dutch relations during the Second World War.

Misinformation has become a worldwide ‘infodemic’, according to the World Health Organization, as exemplified by the COVID-19 anti-vaccination movement, which was endorsed by 33% of United States inhabitants in 2020. In this powerful book, van der Linden analyses why everyone is susceptible to misinformation, how falsehoods spread and how to “inoculate” ourselves through “prebunking” — pre-emptively exposing people to a weakened dose of misinformation so that they can learn to “fend off its manipulative tactics”. US President Joe Biden, for instance, made use of pre-bunking in early 2022 when he issued a public warning of a possible Russian propaganda video showing a fake Ukrainian attack on Russian territory.

Protests against the COVID-19 vaccine in London in June 2023, fuelled by online misinformation.Credit: Loredana Sangiuliano/SOPA Images/LightRocket/Getty

Wasteland Oliver Franklin-Wallis Simon & Schuster (2023)

“Unlike people, garbage doesn’t lie,” writes journalist Oliver Franklin-Wallis in his disturbingly vivid journey through the waste created in various countries. It opens at a British materials-recovery facility, where people in hard hats and high-visibility vests pick through refuse and channel valuable bottles, cardboard and aluminium cans into sorting chutes. Later, he visits a giant landfill near New Delhi, India: 65 metres high, and known to locals as Mount Everest. Here, he meets waste pickers climbing to retrieve valuables to sell, the main occupation of up to 20 million people around the world: “In one man’s trash, another’s treasure.”

Most UK companies refused to show him their landfill sites, which seems strange given that London, New York and San Francisco in California are built partly on garbage. As for plastics, neither manufacturers nor consumers admit the extent of the disposal problem. Instead of discussing ‘plastics’, both groups should understand properties of specific polymers, he recommends, and encourage use of those that are relatively recyclable.

Above all, the book concludes, the key is to “buy less stuff” regardless of advertising: “the three Rs of waste reduction — Reduce, Reuse, Recycle — are not just a catchy slogan, but are actually arranged in order of effectiveness.”

Waste pickers in New Delhi scour rubbish mounds for valuables.Credit: Ravi Choudhary/Hindustan Times/Getty

Psychonauts Mike Jay Yale Univ. Press (2023)

A satirical caricature of a lecture at London’s Royal Institution, drawn in 1802 by James Gillray, is entitled ‘Scientific Researches! — New Discoveries in Pneumaticks! — or — an Experimental Lecture on the Powers of Air’. It shows two lecturers administering nitrous oxide to a governor of the institution with the help of a pair of bellows, producing an explosive effect inside his elegant breeches. One of the lecturers is the chemist Humphry Davy. When Davy first ingested nitrous oxide in the Pneumatic Institution in Bristol in 1799 and dubbed it laughing gas, he became a scientific hero. So did psychologist William James, who in the late nineteenth century took drugs to investigate mystical experiences.

In 1949, the term ‘psychonaut’ was invented by the author Ernst Jünger in his novel Heliopolis, to describe a rebel who, like Davy and James, “went on voyages of discovery in the universe of his brain”. The term became popular in the psychedelic counterculture of the 1960s, but today it connotes a “renegade” unacceptable to institutional science, notes medical historian Mike Jay.

His exploration of psychonauts is a provocative, highly readable meditation on drug use by scientists, philosophers, writers and artists. “All drugs have the potential to heal or to harm,” he concludes.

The Science of Reading Adrian Johns Univ. Chicago Press (2023)

In 1955, Why Johnny Can’t Read — a bestselling book by Rudolf Flesch about the science and pedagogy of reading — provoked controversy when it was published in the United States. The then-fashionable ‘whole word’ method of teaching meant that a child learnt words from their context, like a baby learning to talk. But Flesch claimed that the approach was inferior to the earlier ‘phonics’ method, whereby the child was trained to analyse words’ spelling.

This debate is far from over, “because reading is such a difficult process to understand”, confesses historian of information Adrian Johns. His intriguing analysis discusses the experimental study of reading, beginning in the 1880s.

Johns notes that the field has long been caught in a contradiction. Scientists have used machines such as the tachistoscope (which displays images for short periods of time) and the eye-movement camera to measure almost imperceptible quantities, such as the jumps and momentary pauses of a reader’s eye scanning prose, without being able to explain their psychological significance. Yet “in their grander moments”, they have convinced themselves that “civilization itself depended on those measurements”.

Today, despite the use of scanners to measure brain activity, the reading process remains mostly imponderable.

Viruses Marilyn J. Roossinck Princeton Univ. Press (2023)

Viruses were unknown to Charles Darwin. They were discovered to be a source of infection only after his death, in the 1890s, and were named with the Latin for ‘poison’. Today, in the world of COVID-19, everyone is keenly aware of their impact and many are conscious of their complex structures — yet their definition remains tricky.

The Oxford English Dictionary defines a virus as “typically smaller than a bacterium” and consisting of a nucleic acid “surrounded by a protein coat”. But as virus ecologist Marilyn Roossinck stresses, many giant viruses are larger than some bacteria, and not all viruses have a protein coat. Moreover, not all are agents of disease; some benefit their hosts by helping to protect them from other microbes or allowing them to function in new ways. Indeed, one section of Roossinck’s infectiously enthusiastic, irresistibly illustrated analysis is entitled ‘The Good Viruses’.

Throughout, she stresses viral complexity, noting that there is no simple answer to the question “Are viruses alive?” Many arguments have been offered for and against, although seldom by virologists. “In general, virologists find their favourite entities fascinating, and whether they are alive or not has little relevance because they certainly impact the lives of everything on Earth.”

Einstein in Time and Space Samuel Graydon John Murray (2023)

Science journalist Samuel Graydon calls his first book a “mosaic biography”, aiming to piece together Albert Einstein’s life from brief but significant shards.

Mostly arranged in chronological order, there are 99 sections — matching the atomic number of einsteinium — beginning with Einstein’s birth in Germany in 1879 and ending with his death in the United States in 1955. Each focuses on a specific moment or aspect of its subject. Some concern Einstein’s science, others his personality; many integrate the two.

Particularly fascinating are Einstein’s “inexplicable, incompatible, insane motivations”, including his addiction to tobacco. His US doctors discouraged him from smoking, but his former doctor from Germany, a fellow refugee from Nazism who at that time was also in the United States, took pity on Einstein and supplied it. “As long as the tobacco was not technically his, Einstein felt entitled to smoke it.” One of his pipes, complete with a hole he chewed in it while trying to give up, is the most popular object in the modern physics collection of the National Museum of American History, notes Graydon.

Overall, this book is illuminating, despite inevitable omissions given its relative brevity, such as Einstein’s relationship with India, and in particular with Indian physicist Satyendra Nath Bose.

Nature 624, 516-518 (2023)

doi: https://doi.org/10.1038/d41586-023-04026-x

Competing Interests

The author declares no competing interests.

ESSAY 19 December 2023

Does quantum theory imply the entire Universe is preordained?

The popular idea that quantum physics implies everything is random and nothing is certain might be as far from the truth as it could possibly be.

By

Eddy Keming Chen

Is cosmic evolution a single track with no choice about the destination?Credit: Getty

Was there ever any choice in the Universe being as it is? Albert Einstein could have been wondering about this when he remarked to mathematician Ernst Strauss: “What I’m really interested in is whether God could have made the world in a different way; that is, whether the necessity of logical simplicity leaves any freedom at all.”

US physicist James Hartle, who died earlier this year aged 83, made seminal contributions to this continuing debate. Early in the twentieth century, the advent of quantum theory seemed to have blown out of the water ideas from classical physics that the evolution of the Universe is ‘deterministic’. Hartle contributed to a remarkable proposal that, if correct, completely reverses a conventional story about determinism’s rise with classical physics, and its subsequent fall with quantum theory. A quantum Universe might, in fact, be more deterministic than a classical one — and for all its apparent uncertainties, quantum theory might better explain why the Universe is the one it is, and not some other version.

In physics, determinism means that the state of the Universe at any given time and the basic laws of physics fully determine the Universe’s backward history and forward evolution. This idea reached its peak with the strict, precise laws about how the Universe behaves introduced by classical physics. Take Isaac Newton’s laws of motion. If someone knew the present positions and momenta of all particles, they could in theory use Newton’s laws to deduce all facts about the Universe, past and future. It’s only a lack of knowledge (or computational power) that prevents scientists from doing so.

Along with this distinctive predictive power, determinism underwrites scientific explanations that come close to the ‘principle of sufficient reason’ most famously articulated by German polymath Gottfried Leibniz: that everything has an explanation. Every state of the Universe (with one obvious exception, which we’ll come to) can be completely explained by an earlier one. If the Universe is a train, determinism says that it’s running on a track, with no option to switch to any other path because different tracks never cross.

Physicists have conventionally liked determinism’s predictive and explanatory power. Others, including some philosophers, have generally been more divided, not least because of how determinism might seem to preclude human free will: if the laws of physics are deterministic, and our actions are just the summation of particle interactions, there seems to be no room for us to freely choose A instead of B, because the earlier states of the Universe will already have determined the outcome of our choice. And if we are not free, how can we be praised or blamed for our actions? Neuroendocrinologist Robert Sapolsky’s 2023 book Determined touches on this fascinating and controversial issue.

Space invaders

The strange behaviours of quantum particles that began to emerge in the twentieth century fundamentally shifted the debate surrounding determinism in physics. The laws of quantum mechanics give only the probabilities of outcomes, which can be illustrated with the thought experiment devised by Austrian physicist Erwin Schrödinger in 1935 (although when he devised it, he was concerned mainly with how the wavefunction represents reality). A cat is trapped in a box with a vial of poison that might or might not have been broken by a random event — because of radioactive decay, for example. If quantum mechanics applied to the cat, it would be described by a ‘wavefunction’ in a superposition of ‘alive’ and ‘dead’. The wavefunction, when measured, randomly jumps to one of the two states, and quantum mechanics specifies only the probability of either possibility occurring. One consequence of the arrival of quantum mechanics was that it seemed to throw determinism out of the window.

But this accepted idea might not be the whole story, as developments in the second half of the twentieth century suggested. The quantum Universe could actually be more deterministic than a classical one, for two reasons. The first is technical. Newton’s laws allow situations in which the past does not determine how things will move in the future. For example, the laws do not provide an upper bound on how much an object can be accelerated, so in theory a classical object can reach spatial infinity in finite time. Reverse this process, and you get what have been called ‘space invaders’ — objects that come from spatial infinity with no causal connection to anything else in the Universe, and which can’t be predicted from any of the Universe’s past states.

In practice, this problem is solved by the universal speed limit, the speed of light, introduced by Einstein’s special theory of relativity. But unruly infinities also plague Einsteinian relativity, which is a classical theory. The equations of general relativity lead to ‘singularities’ of infinite curvature, most notoriously in black holes and at the Big Bang at the beginning of the Universe. Singularities are like gaps in space-time where the theory no longer applies; in some cases, anything can come out of them (or disappear into them), threatening determinism.

Many physicists think that quantum theory can come to the rescue by removing such singularities — for example, by converting the Big Bang into a ‘Big Bounce’, with a Universe that continues to evolve smoothly on the other side of the singularity. If they are right, a theory of ‘quantum gravity’ that fully unifies quantum theory, which predicts the behaviour of matter on the smallest scales, and Einstein’s relativity, which encapsulates the large-scale evolution of the Universe, will smooth out the gaps in space-time and restore determinism.

Space-time ‘singularities’ inside black holes could threaten a deterministic cosmic order.Credit: ESO/SPL

But there is a deeper reason why the quantum Universe might be more deterministic, to which Hartle’s scientific legacies are relevant. With US physicist Murray Gell-Mann, Hartle developed an influential approach to quantum theory, called decoherent histories[1]. This attempted to explain the usefulness of probabilistic statements in quantum physics, and the emergence of a familiar, classical realm of everyday experience from quantum superpositions. In their picture, the wavefunction never randomly jumps. Instead, it always obeys a deterministic law given by Schrödinger’s equation, which characterizes the smooth and continuous evolution of quantum states. In this respect, it is similar to US physicist Hugh Everett III’s popular ‘many worlds’ interpretation of quantum mechanics, which proposes that the quantum Universe splits into different branches according to the possibilities encoded in the wavefunction whenever anything is measured[2]. In what follows I assume, as Everett did, that the Universe can be completely described by a quantum wavefunction with no ‘hidden’ variables that operate on a more fundamental level.

Into the quantum cosmos

With Stephen Hawking, Hartle went on to become one of the founders of quantum cosmology, which applies quantum theory to the entire Universe. In a classical Universe, there is freedom in choosing how it all started. Even setting aside the extreme situations mentioned earlier, classical mechanics is deterministic merely in that it lays down many possible evolutionary histories for the Universe, and offers conditional statements about them: if this happens, then that must happen next. To return to the train analogy, a deterministic theory does not, by itself, say why the train is on any one given track out of many: why it is going from A to B via C, rather than from X to Y via Z. We can go back to earlier states to explain the current state, and do that all the way back to the initial state — but this initial state is not explained by anything that precedes it. Ultimately, standard determinism fails to fully satisfy Leibniz’s principle of sufficient reason: when it comes to the initial state, something remains without an explanation.

This failure is not just philosophical. A complete theory of the Universe should predict the phenomena we observe in it, including its large-scale structure and the existence of galaxies and stars. The dynamic equations we have, whether from Newtonian physics or Einsteinian relativity, cannot do this by themselves. Which phenomena show up in our observations depend sensitively on the initial conditions. We must look at what we see in the Universe around us, and use this information to determine the initial condition that might have given rise to such observations.

A theory that specifies deterministic laws of both the Universe’s temporal evolution and its exact initial condition satisfies what English physicist Roger Penrose called ‘strong determinism’ in his 1989 book The Emperor’s New Mind. This is, according to Penrose, “not just a matter of the future being determined by the past; the entire history of the universe is fixed, according to some precise mathematical scheme, for all time”. Let us say that a Universe is strongly deterministic if its basic laws of physics fix a unique cosmic history. If determinism provides a set of non-crossing train tracks, without specifying which one is being used, then strong determinism lays down a single track that has no choice even about where it starts.

A universal wavefunction

Strong determinism is hard to implement in classical physics. You might consider doing it by specifying the initial condition of the Universe as a law. But although the dynamical laws of classical physics are simple, the Universe itself is complex — and so its initial condition must have been, too. Describing the precise positions and momenta of all the particles involved requires so much information that any statement of the initial condition is too complex to be a law.

Hartle suggested[3] that quantum mechanics can solve this complexity problem. Because a quantum object’s wavefunction is spread out across many ‘classical’ states (cat alive or cat dead, for instance), you could propose a simple initial condition that includes all the complexities as emergent structures in the quantum superposition of these states. All the observed complexities can be regarded as partial descriptions of a simple fundamental reality: the Universe’s wavefunction. As an analogy, a perfect sphere can be cut into many chunks with complicated shapes, yet they can be put back together to form a simple sphere.

In 1983, Hartle and Hawking introduced[4] one of the first (and highly influential) proposals about the quantum Universe’s initial state. Their ‘no boundary’ wavefunction idea suggests that the ‘shape’ of the Universe is like that of a shuttlecock: towards the past, it rounds off smoothly and shrinks to a single point. As Hawking said in a 1981 talk on the origin of the Universe in the Vatican: “There ought to be something very special about the boundary conditions of the Universe, and what can be more special than the condition that there is no boundary?”

In this perspective, the quantum Universe has two basic laws: a deterministic one of temporal evolution and a simple one that picks an initial wavefunction for the Universe. Hence, the quantum Universe satisfies strong determinism. The physical laws permit exactly one cosmic history of the Universe, albeit one described by a wavefunction that superposes many classical trajectories. There is no contingency in what the Universe as a whole could have been, and no alternative possibility for how it could have started. Every event, including the first one, is explained; the entire wavefunction of the Universe for all times is pinned down by the laws. The probabilities of quantum mechanics do not exist at the level of the basic physical laws, but can nonetheless be assigned to coarse-grained and partial descriptions of bits of the Universe.

This leads to a more predictive and explanatory theory. For example, the no-boundary proposal makes predictions for a relatively simple early Universe and for the occurrence of inflation — a period of rapid expansion that the Universe seems to have undergone in its first instants.

There are still many wrinkles to this proposal, not least because some studies have shown that, contrary to initial expectations, the theory might not single out a unique wavefunction for the Universe[5],[6]. But studies in quantum foundations — research that is mostly independent from that of quantum cosmology — could offer yet another method for implementing strong determinism. Several researchers have considered the controversial idea that quantum states of closed systems, including the Universe, need not be restricted to wavefunctions, but instead can come from a broader category: the space of density matrices[7]–[10].

The ultimate theory

Density matrices can be thought of as ‘superpositions of superpositions’, and they provide extra options for the initial condition of the Universe. For example, if we have reasons to adopt the ‘past hypothesis’ — the idea, which seems likely, that the Universe began in a low-entropy state (and its entropy has been increasing steadily since) — and that this theory corresponds to a set of wavefunctions, then we can choose a simple density matrix that corresponds to the uniform mixture of that set. As I have argued[10], if we regard the density matrix as the initial state of the Universe and accept that it is specified by a law, then this choice, together with the deterministic von Neumann equation (a generalization of Schrödinger’s equation), can satisfy strong determinism. However, in this case, the laws fix a cosmic history of a quantum Universe that has many evolving branches — a ‘multiverse’.

So how deterministic is the Universe? The answer will depend on the final theory that bridges the divide between quantum physics and relativity — and that remains a far-off prospect. But if Hartle is right, the story of the rise and fall of determinism until now might be the reverse of the conventional tale. From a certain perspective, the quantum Universe is more deterministic than a classical one, providing stronger explanations and better predictions. That has consequences for humans, too, because that makes it harder to appeal to quantum theory to defend free will[11]. If the quantum Universe is strongly deterministic, then there is no other path to make the Universe than the way it is. The ultimate laws of the quantum cosmos might tell us why it is this one.

Nature 624, 513-515 (2023)

doi: https://doi.org/10.1038/d41586-023-04024-z

References

1. Gell-Mann, M. & Hartle, J. B. in Complexity, Entropy and the Physics of Information (ed. Zurek, W. H.) (Addison Wesley, 1990).

2. Buchanan, M. Nature 448, 15–17 (2007).

3. Hartle, J. B. in Boundaries and Barriers: On the Limits of Scientific Knowledge (eds Casti, J. L. & Karlqvist, A.) 116–147 (Addison Wesley, 1996).

4. Hartle, J. B. & Hawking, S. W. Phys. Rev. D 28, 2960–2975 (1983).

5. Calcagni, G. Classical and Quantum Cosmology (Springer, 2017).

6. Lehners, J.-L. Phys. Rep. 1022, 1–82 (2023).

7. Dürr, D., Goldstein, S., Tumulka, R. & Zanghì, N. Found. Phys. 35, 449–467 (2005).

8. Wallace, D. The Emergent Multiverse: Quantum Theory according to the Everett Interpretation (Oxford Univ. Press, 2012).

9. Allori, V., Goldstein, S., Tumulka, R. & Zanghì, N. Br. J. Philos. Sci. 65, 323–352 (2014).

10. Chen, E. K. Br. J. Philos. Sci. 72, 1155–1183 (2021).

11. Kane, R. in Quantum Physics Meets the Philosophy of Mind: New Essays on the Mind-Body Relation in Quantum-Theoretical Perspective (eds Corradini, A. & Meixner, U.) 163–182 (De Gruyter, 2014).

Competing Interests

The author declares no competing interests.

Opinion

- To build a better world, stop chasing economic growth

- Devise an ethical open-access publishing model

COMMENT 20 December 2023

To build a better world, stop chasing economic growth

The year 2024 must be a turning point for shifting policies away from gross domestic product and towards sustainable well-being. Here’s why and how.

By

Robert Costanza

Protesters in New York City in September joined thousands of others around the world in urging governments to act on climate change. Credit: Eduardo Munoz/Reuters

The past year has given many of us reason to pause. We are losing in a race to prevent planetary tipping points — the climate is changing faster than expected, and humanity has already breached six of the nine sustainable planetary boundaries (for biodiversity loss; climate, freshwater and land-system change; biogeochemical flows; and novel entities)[1]. Summer Antarctic sea ice shrank to its lowest recorded extent in 2023 (see go.nature.com/4f86req), a year that is on track to be the warmest on record (see go.nature.com/4f9ykdj).

People around the world recognize that life is not getting any better. As wars rage, runaway inequality and political polarization are eroding societies’ sense of cohesion. Eight individuals owned more than the poorest 50% of the world’s population, according to an Oxfam report in 2017[2]. Levels of anxiety, depression and burnout are rocketing. Full-time employees are unable to pay rent and must turn to extra part-time work to make ends meet, while employers cut staff and increase workloads.

Crises are now normal in this global economic system that depletes natural and social capital, energy and time in the name of economic growth at all costs.

But, looking to 2024, I’m hopeful that the world can turn in a better direction. For example, a meeting I attended in May on sustainable prosperity particularly buoyed my spirits — and, in my view, signalled a tipping point in thinking and governance. The Beyond Growth conference at the European Parliament attracted more than 2,500 participants in person, as well as 2,000 online. It was sponsored by the European Commission and the Club of Rome (a non-profit organization fostering research and action around pressing global issues).

In a stirring opening address, Ursula von der Leyen, president of the European Commission, said that governments must stop misusing growth of gross domestic product (GDP) as their goal and instead move swiftly and urgently to sustainable well-being within planetary boundaries. She got a standing ovation. Agata Meysner, the young leader of Generation Climate Europe, a coalition of climate and environmental networks across the bloc, concluded the event with a call to join the “movement of movements” to create a new economy based on sustainable prosperity, justice and sufficiency. Everyone rose to their feet.

And an increasing number of organizations and movements dedicated to overcoming our addiction to GDP growth are working together to do just that, including through networks such the Wellbeing Economy Alliance (which also supported the conference). The pursuit of GDP growth at all costs is an outdated paradigm that claims that all people want is more income and consumption with no limits. It assumes that the market economy can grow forever, that massive inequality is justified to provide incentives to promote growth, and that efforts to address climate and other environmental and social problems must not interfere with growth. It supposes that growth is the solution to all ills. It isn’t.

As the European conference emphasized, GDP was never designed to measure societal well-being — only market production and consumption. GDP says nothing about the distribution of income, unpaid work or damages to natural or social capital. The misuse of GDP as a policy goal is driving societies towards an unsustainable future that benefits an increasingly small proportion of the population while impoverishing the vast majority[2].

Researchers must help to provide alternatives. Here’s how.

Design better measures of societal well-being

Hundreds of indicators of societal well-being are already in use, including by the United Nations, the World Bank, the Organisation for Economic Co-operation and Development (OECD), non-governmental organizations, countries and academics. Examples include the Genuine Progress Indicator; the OECD Better Life Index; and annual surveys of life satisfaction[3]. To become a societal goal used by all, and to displace GDP, the world must settle on a new indicator. Broad consensus is needed on what should be included.

For example, it is not just income that matters, but also the ways in which it is distributed. The costs of environmental and social degradation must be included, as should contributors to well-being that are unconnected to income — such as our relationships and communities, good governance, the ability to participate in decision-making and ecosystem services provided by the natural environment. Several research initiatives are beginning to address these issues (including one I am involved with, called MERGE, which is funded by the European Union).

The Beyond Growth conference in Brussels in May 2023 concluded with a call to create a sustainable and just economy.Credit: Ida Kubiszewski

Model the complex dynamics of the economic system

Interactions between social, economic and natural elements of societal well-being must be better modelled and future projections developed to assess their sustainability. National examples include the EUROGREEN model, which has been applied to France, and the LowGrow model of the Canadian ecological economy.

The Earth4All model of the Club of Rome takes a global perspective. It explores two scenarios towards 2050. The first — known as business as usual, or too little too late — looks at what will happen if the world continues current trends of increasing inequality, climate disruption and decreasing well-being, even as GDP continues to rise. By contrast, the second scenario, called giant leap, shows how investing in five areas — renewable energy, regenerative food, reducing inequality, eliminating poverty and enabling empowerment — could ensure sustainable, prosperous and equitable well-being for humans and the rest of nature.

For example, rather than rewarding the fossil-fuel sector with enormous subsidies to maintain economic growth at the expense of climate and social disruption, new policies would focus on moving away from fossil fuels. At the same time, policies to achieve the other four turnaround areas would be implemented to enable sustainable well-being, regardless of the effects on GDP.

Develop policies to support sustainable well-being

The EU and the Wellbeing Economy Governments (WEGo) group that includes Scotland, New Zealand, Wales, Finland, Canada and Iceland have begun to implement measures of sustainable well-being and policies to achieve them. Researchers need to analyse these plans and offer lessons to help other nations adopt them. What are these policies? There are many versions, but an open letter I signed in May — together with more than 400 leading economists, scientists, policymakers and activists — provides starting points in four areas. These are biocapacity (protecting the planet), fairness (a more equal society), well-being for all (basic services and rights) and active democracy (see ‘Beyond growth policies’ and go.nature.com/47z3v6s).

Beyond growth policies

An open letter signed by more than 400 experts in May suggested legislation based on four principles that would overcome the world’s addiction to growth in gross domestic product (see also go.nature.com/47z3v6s; suggestions have been edited for clarity).

Biocapacity. Policies could include phasing out fossil fuels, limits to extraction of raw materials and measures for nature protection and restoration to ensure healthy and resilient soils, forests and marine and other ecosystems. Examples are a fossil-fuel non-proliferation treaty and a resource justice and resilience act, which would involve a binding target to reduce material footprints and real, area-based nature restoration.

Fairness. Fiscal instruments could be used to foster a more equal society by eradicating income and wealth extremes, as well as super-profits. These might include a carbon-wealth tax and setting of minimum and maximum incomes.

Well-being for all. An improved, ecologically sensitive welfare state would secure people’s access to essential infrastructures. This might be achieved through the provision of universal basic services (including the human rights to health, transport, care, housing, education and social protection), job guarantees and price controls for essential goods and services.

Active democracy. Citizen assemblies could be set up with mandates to formulate socially acceptable sufficiency strategies and strengthen policies. These would be based on ecological limits, fairness and well-being for all, and include a stronger role for trade unions. Examples include local-needs forums, climate conventions and participatory budgeting.

Specific policies will have to be tailored for a broad range of contexts. They will need to overturn and overcome opposition from vested interests that are maintaining the current system, including billionaires, the fossil-fuel sector, big pharma, the defence industry and industrial agriculture. For example, perverse subsidies for damaging practices across these sectors must be removed. Collective actions through ‘movements of movements’ will be essential for bypassing lobbying and misinformation.

Build a shared vision

Constructing a sustainable world where well-being is prioritized must be a key goal for 2024 and beyond. At the societal scale, people need a positive shared vision of goals that can achieve sustainable well-being[4]. To motivate change, techniques might be borrowed from therapies for addictive behaviours, such as ‘motivational interviewing’[5], which engages people who have addictions in a positive discussion of their life goals.

The UN Sustainable Development Goals (SDGs), which were agreed by all countries in 2015, are one step in the process. But they have not been fully embraced by governments and are largely unknown to the public. To change that, researchers, activists and policymakers need to develop creative ways to engage people. For example, those in the arts and humanities could create positive visions of what life would look like in a world where the SDGs have been achieved. This could help to motivate the movement of movements and other changes needed to overcome the addiction to GDP growth.

People often fear that such transformations will require sacrifices. In the short term, change is difficult, and addictions are powerful. But in the long run, it is a huge sacrifice of our personal and societal well-being to continue down the business-as-usual path. Sustainable well-being can improve the lives of everyone, and protect the biodiversity and ecosystem services on which we all depend. In the coming year, let’s continue to build the shared vision of the world we all want, and accelerate progress towards it.

Nature 624, 519-521 (2023)

doi: https://doi.org/10.1038/d41586-023-04029-8

References

1. Richardson, K. et al. Sci. Adv. 9, adh2458 (2023).

2. Hardoon, D. An Economy for the 99%. Oxfam Briefing Paper https://doi.org/10.21201/2017.8616 (2017).

3. Costanza, R. et al. Nature 505, 283–285 (2014).

4. Costanza, R. Addicted to Growth: Societal Therapy for a Sustainable Wellbeing Future (Routledge, 2023).

5. Miller, W. R. & Rollnick, S. Motivational Interviewing: Helping People Change 3rd edn (Guilford, 2012).

Reprints and permissions

Competing Interests

The author declares no competing interests.

CORRESPONDENCE 19 December 2023

Devise an ethical open-access publishing model

By

Mark A. Elgar

Scholarly publishing revenue has conventionally come from library subscriptions paid for mainly by publicly funded institutions, and so is underwritten by the taxpayer. In my view, such institutions should also pay for their open-access publications. Recognizing who ultimately pays for scholarly publishing therefore raises ethical concerns (see Nature 623, 472–473; 2023).

The open-access business model requires authors to pay article-processing charges (APCs). This stimulates a keen interest in the variation in APCs, especially for authors expected to dip into their research grants. These frustrated colleagues frequently write to me, as editor-in-chief of a leading open-access journal, demanding that the publisher waive the APCs. I doubt they have ever demanded that publishers of subscription journals provide a free subscription.

But taxpayers underwrite scholarly publishing for publicly funded institutions, irrespective of the business model, and so the institutions — rather than individual researchers — should pay the APCs. And those institutions need to collaborate with publishers to produce a financially viable, ethical open-access scholarly publishing model. Lobbying your vice-chancellor or president, rather than complaining to journal editors, should facilitate the process.

Nature 624, 523 (2023)

doi: https://doi.org/10.1038/d41586-023-04056-5

Competing Interests

M.A.E. receives an honorarium from Frontiers Media Ltd as Field Chief Editor of Frontiers in Ecology and Evolution.

Work

- The scientific workplace in 2023

- How I’m protecting Clanwilliam sandfish

CAREER FEATURE 18 December 2023

The scientific workplace in 2023

India’s first Moon landing and a welcome return to Horizon Europe for UK researchers loomed large in an eventful year for working scientists around the world.

By

Chris Woolston

Find a new job

People in India celebrate the successful landing of Chandrayaan-3 near the Moon’s south pole in August. Credit: Manish Swarup/AP via Alamy

In a year of great scientific progress and discovery, researchers around the world dealt with a wide range of issues — financial, political, social and otherwise — that affected their ability to do their work.

As cost-of-living pressures continued to rise and many governments cut funding for science, researchers organized and protested in response. The year will be remembered for the explosive rise of generative artificial intelligence (AI), but it was also a year of backtracking on the United Nations’ Sustainable Development Goals and net-zero carbon commitments. Geopolitical tensions continue in many parts of the world, including the war between Russia and Ukraine, conflict triggered by Hamas attacks on Israel in October and Israel’s bombardment of Gaza. India’s Chandrayaan-3 mission reached the Moon and the United Kingdom rejoined Horizon Europe, the European Union’s €95-billion (US$102-billion) research programme, after a two-year hiatus.

Here is a collection of some of the top stories that, taken together, paint a picture of the scientific workplace in 2023, including steps that researchers took to advocate for themselves and their profession.

Speaking out, taking action

Nature’s second global survey of postdoctoral researchers revealed deep, ongoing concerns about pay, working conditions and career development, findings that amplified calls for reform. A separate survey of postdocs conducted by the US National Postdoctoral Association confirmed widespread dissatisfaction with working conditions and job prospects, with almost 95% of the 366 respondents saying low pay was negatively affecting their personal and professional lives.

Postdocs at the US National Institutes of Health (NIH), the nation’s main funder of biomedical and public-health research, warned that cost-of-living pressures are gutting its workforce, and thousands of early-career NIH researchers voted to unionize for the first time. In Canada, PhD students and postgraduates staged a mass walkout over low pay. Hundreds of researchers in Germany protested to call for improved working conditions for postdocs and others in academia.

In June, the issue of race-conscious university admissions went before the US Supreme Court.Credit: Chip Somodevilla/Getty

In the United States, researchers at the University of California ended a historic strike. But research-group leaders around the country worried about how to pay for workers’ salary hikes.

Some fed-up early-career researchers have found a subtler way to express dissatisfaction through ‘quiet quitting’ — by no longer performing unpaid or under-appreciated tasks.

Research misconduct

Concerns about research integrity and how investigations are handled continued to overshadow the scientific enterprise. Ram Sasisekharan, a bioengineer at Massachusetts Institute of Technology in Cambridge, is having to rebuild his team after a years-long probe that eventually cleared him of wrongdoing. Neuroscientist Marc Tessier-Lavigne stepped down as president of Stanford University in California amid criticism over lax oversight of his laboratory, a case that could offer lessons for other lab leaders. A US judge ruled that Charles Lieber would not have to face any extra prison time for lying about ties to China while working as a chemist at Harvard University in Cambridge, Massachusetts.

In March, bioengineer Ram Sasisekharan was cleared of wrongdoing.Credit: Bryce Vickmark

Australia, a country that lacks an independent body to investigate scientific misconduct, started re-evaluating its system of oversight after several high-profile instances of scientific fraud. And with the advent of the AI chatbot ChatGPT, scientific sleuths have been working to spot its dishonest use in papers.

Axed funding

Scientists in many countries spoke out about government decisions and actions that threaten to undermine the research enterprise. Australia’s decision to slash funding for its Antarctica programme — an important source of climate-change data — was described as a ‘terrible blow for science’. The Japanese government drew criticism over a plan to privatize its influential science council. In Mexico, thousands of researchers protested against a new law that consolidated government control over the country’s science. And Swedish researchers decried their government’s decision to axe funding for research on international development. The Indian government’s decision to suspend foreign funding for the New Delhi-based Centre for Policy Research — an influential think tank — drew condemnation from international researchers, including many with a long history of collaboration with the centre. Many feared it could muzzle independent scrutiny of policy in India. Cochrane, a global independent network that champions evidence-based health care, battled UK funding cuts and closures.

Partnerships and tensions

Scientific collaboration across borders improved in some ways in 2023. The United Kingdom rejoined the Horizon Europe research programme, a move widely celebrated as a welcome sign of unity and progress following tensions triggered by the 2016 Brexit vote.

In another example of cooperation, the United States agreed an extension to a non-binding research pact with China, first signed in 1979 and renewed every five years since then. Some US lawmakers had said the agreement poses a threat to national security and called for it to be scrapped. A US State Department spokesperson said the six-month extension was agreed to amend and strengthen the deal.

But the two countries remain rivals. For the first time, China topped the United States as the leading producer of natural-science papers, as measured by the Nature Index database, which tracks research outputs published across 145 leading medical and natural-sciences journals.

The United Kingdom hosted a global summit on AI safety, one of the hallmark issues of 2023, drawing delegates from 27 nations and representatives from the technology industry.

Separately, UK Prime Minister Rishi Sunak announced that the country was delaying a ban on new petrol and diesel car sales from 2030 to 2035. A phase-out of new gas-fired boilers for heating homes by 2030 was pushed back by the same amount. The announcement followed one earlier in the year of new exploration licences for oil and gas companies. Also, a plan to expand capacity for offshore wind energy was put on hold. The NIH announced, and later upheld, a plan to more heavily scrutinize grants awarded to foreign scientists, a move that drew condemnation from many researchers.

Diversity challenges

Science continued to grapple with issues of diversity, a still-fraught topic marked by both progress and setbacks. Following a US Supreme Court decision, US universities moved to end ‘race-conscious’ admissions, a shift with implications for diversity on campus and ultimately in the scientific workforce. Moves by US universities to defund diversity programmes drew criticism from academics.

Scientists petitioned the University of California, Los Angeles, to reinstate ecologist Priyanga Amarasekare, who was suspended after speaking out about discrimination. Similarly, supporters of gender-equality researcher Susanne Täuber petitioned the University of Groningen in the Netherlands for her reinstatement — Täuber had been sacked after criticizing the university’s failure to follow its own equality policy.

After receiving much criticism for declining to include questions about sexual orientation on workforce surveys, the US National Science Foundation, the nation’s main federal research funder, announced plans to track sexual orientation and gender identity demographics when it releases its yearly census of all recipients of research doctorates at US institutions in 2024, pending the results of a pilot survey conducted in July 2023. The NIH also faced calls to make its policies and procedures more inclusive of people with disabilities by changing what critics see as its ‘ableist’ mission statement.

A study[1] of more than one million papers published in more than 500 journals between 2001 and 2020 found signs of racial inequalities in publishing, including under-representation of members of minority ethnic groups on editorial boards, and lower citation rates and longer review periods for their papers compared with those by white authors. The largest-ever study[2] of journal editors found a strong gender gap on editorial boards, with only 14% of journal editors being women and only 8% of editors-in-chief being women. But a third investigation[3] found that women in some research fields are 3–15 times more likely than men with similar levels of citations and publications to be elected to prestigious US scientific academies.

Sexual harassment in the spotlight

Science’s ongoing reckoning with bullying, harassment and incidents of sexual assault incidents made some headlines. Joshua Tewksbury, director of the Smithsonian Tropical Research Institute in Panama, described changes in protocols and culture that the institute made after incidents of harassment came to light. Researchers urged UK universities to improve their processes for handling allegations of sexual assault, and the University of Oxford took a step towards avoiding potential conflicts of interest and abuses of power by banning intimate relationships between students and staff.

Wins for science

Indian researchers had some reasons for optimism. The touchdown of the Chandrayaan-3 mission’s lander on the Moon was hailed as a landmark achievement for India and a great advance in international efforts to understand Earth’s nearest neighbour. Plans to create a billion-dollar National Research Foundation raised hopes that the country could boost research across its thousands of universities, colleges, institutes and laboratories.

In the United Kingdom, modifications to the country’s Research Excellence Framework were seen by many as a promising step towards better assessment of scientific productivity, but others warned that the changes could cause more problems than they solve. Researchers in Australia welcomed a move to limit the government’s power to veto grant approvals. And the European Union invested €10 million to address a ‘brain drain’ that has seen early-career scientists seeking jobs elsewhere.

The publishing landscape

Academic publishers continued to grapple with a shifting landscape and the growth of open-access publishing. Members of the editorial boards at the neuroscience journal NeuroImage and its companion journal NeuroImage: Reports quit in protest at what they saw as high article-processing charges.

For scientists, simply trying to adhere to journal guidelines could be wasting millions of dollars’ worth of their time every year, an investigation[4] suggested. Another study[5] found that researchers who don’t speak fluent English get little help from journals. Scientists in Tanzania, however, were offered a direct yet controversial incentive by the government: a US$22,000 bonus if their work is published in international journals.

In August, the family of Henrietta Lacks reached a settlement over how her cells were used.Credit: Jonathan Newton/The Washington Post via Getty

Science, justice and conflict

Researchers in Sudan had to flee a violent military conflict that spilled over into hospitals and universities. Hamas attacks on Israel and Israel’s bombardment of Gaza emptied labs, with many Israeli researchers called up for military service, and damaged university buildings in Gaza. In the United States, the NSF invested millions in an effort to unite Indigenous knowledge with Western science. And a ground-breaking settlement between the biotechnology company Thermo Fisher Scientific and the family of Henrietta Lacks, a Black woman who had cells taken from her and used for research without consent more than 70 years ago, has implications for the use of human tissue in research. Stephen Sodeke, a bioethicist at Tuskegee University in Alabama, said that if a person consents to a surgical procedure, they should “have the legal right to decide whether to allow the use or not of cells derived” from it for research.

Whatever’s in store for science 2024, Nature will ensure that the scientific workplace receives the attention, scrutiny and coverage that it deserves.

Nature 624, 689-691 (2023)

doi: https://doi.org/10.1038/d41586-023-04030-1

References

1. Liu, F., Rahwan, T. & AlShebli, B. Proc. Natl Acad. Sci. USA 120, e2215324120 (2023).

2. Liu, F., Holme, P., Chiesa, M., AlShebli, B. & Rahwan, T. Nature Hum. Behav. 7, 353–364 (2023).

3. Card, D., DellaVigna, S., Funk, P. & Iriberri, N. Proc. Natl Acad. Sci. USA 120, e2212421120 (2023).

4. Clotworthy, A. et al. BMC Med. 21, 172 (2023).

5. Arenas-Castro, H. et al. Preprint at EcoEvoRxiv https://doi.org/10.32942/X2NS3K (2023).

WHERE I WORK 18 December 2023

How I’m protecting Clanwilliam sandfish

Cecilia Cerrilla’s PhD project is to protect a tiny species of fish from predatory bass.

By

Jack Leeming

Cecilia Cerrilla is a PhD candidate at the University of Cape Town, South Africa. Credit: Barry Christianson for Nature

To lay eggs, Clanwilliam sandfish (Labeo seeberi) swim upstream to gentler, shallower tributaries of the Doring/Olifants river system in the Western Cape and Northern Cape provinces of South Africa. Local people say it was once an epic event: the water seemed to change colour as thousands of sandfish migrated upstream. Now, however, those numbers have shrunk to dangerous levels.

That’s where I come in. My PhD at South Africa’s University of Cape Town is a collaboration with the Saving Sandfish project, run by a non-governmental organization called the Freshwater Research Centre in Cape Town — so I am also a conservationist.

Human activity, climate change and thirsty invasive plants, are draining the rivers. As newly hatched sandfish try to swim downstream, they now get stuck in shallow pools, making them vulnerable to predators such as bass species introduced into the river system for sport fishing in the 1900s.

The sandfish population has declined by more than 90% since we began keeping count in 2013. They’re now classified as endangered by the International Union for the Conservation of Nature.

When the river starts to dry up, we scoop out young sandfish and put them into buckets of water, then move them by truck to one of six pre-prepared nurseries donated by local people. In this photo, I’m lifting a fish from one of those reservoirs. The support from local people is amazing.

Once the sandfish are large enough to be less threatened by the bass, we return them to the wider ecosystem.

We’re at an early stage, but the data so far show the project has been successful. We’ve rescued some 36,000 young sandfish over the past three years and have released almost 3,000. Last year, we got 77 readings from fish coming back from the group we released into the wild. This year, 222 have come back so far. I’m looking forward to adding to those numbers next year.

Nature 624, 694 (2023)

doi: https://doi.org/10.1038/d41586-023-04031-0

Research

- 2D materials ratchet up biorealism in computing

- Homogenizing out-of-plane cation composition in perovskite solar cells

- Autonomous chemical research with large language models

- Biological carbon pump estimate based on multidecadal hydrographic data

- Human mobility networks reveal increased segregation in large cities

- Indigenous Australian genomes show deep structure and rich novel variation

- The landscape of genomic structural variation in Indigenous Australians

- Lung dendritic-cell metabolism underlies susceptibility to viral infection in diabetes

- Structural insights into intron catalysis and dynamics during splicing

NEWS & VIEWS FORUM 20 December 2023

2D materials ratchet up biorealism in computing

A transistor made from atomically thin materials mimics the way in which connections between neurons are strengthened by activity. Two perspectives reveal why physicists and neuroscientists share equal enthusiasm for this feat of engineering.

By

Frank H. L. Koppens,

James B. Aimone &

Frances S. Chance

THE PAPER IN BRIEF

• The brain offers ample inspiration for computer engineers, but such ‘neuromorphic’ devices can be disadvantaged by huge power consumption, limited endurance and considerable variability.

• One of the challenges associated with optimizing these devices involves ascertaining which brain characteristics to emulate.

• In a paper in Nature, Yan et al.[1] report a type of synaptic transistor — a device named after its similarities to neural connections known as synapses — that maximizes performance through a ratcheting mechanism that is reminiscent of how neurons strengthen their synapses.

• The transistor could enable energy-efficient artificial-intelligence algorithms, and reproduce some of the many sophisticated behaviours of the brain.

FRANK H. L. KOPPENS: A new twist on synaptic transistors

At the heart of Yan and colleagues’ innovation lies the unusual behaviour of electrons that arises when materials of single-atom thickness are stacked together and then twisted relative to each other. The materials in question are bilayer graphene, which comprises two stacked layers of carbon atoms, sandwiched together by two layers of the dielectric material hexagonal boron nitride (hBN)[2]. Both of these materials have hexagonal crystal structures, but the spacing between their atoms differs slightly. The overlapping hexagonal patterns create regions of constructive and destructive interference, resulting in a larger-scale pattern known as a moiré lattice.

The moiré pattern modifies how electrons are distributed in bilayer graphene: it localizes them periodically throughout the crystal lattice (Fig. 1a). Electrons in the top layer of graphene are affected more by this periodic modulation because the crystal structure of this layer is aligned with that of the hBN above it, and this essentially makes the electrons immobile. By contrast, the hBN below the bottom graphene layer is rotated out of alignment with the graphene, resulting in a weaker electronic modulation[3]. The electrons in this layer are therefore mobile, and they contribute to the current flow.

Figure 1 | A transistor that imitates a biological synapse. Yan et al.[1] built a device comprising two layers of graphene (each a single sheet of carbon atoms) and the dielectric material hexagonal boron nitride (hBN). a, The device is called a moiré synaptic transistor, because it shares similarities with synaptic connections between neurons, and because a ‘moiré’ pattern forms between the overlapping hexagonal crystal structures of the top layer of graphene and hBN. This pattern localizes electrons in the top graphene layer, but those in the bottom layer remain mobile. Applying a voltage pulse to the top gate (a component that regulates the number of electrons in the graphene system; not shown) results in a ratchet effect, through which an electric current is increased with successive pulses. b, This effect is reminiscent of the way in which repeated electrical stimulation can strengthen synapses by enriching protein complexes called AMPA receptors, leading to improved neurotransmitter efficacy and increased ion flow.

This asymmetry between layers makes the transistor function like a kind of ratchet, controlling the flow of mobile electrons and regulating the device’s electrical conductance, which is analogous to synaptic strength. The ratchet is controlled by two ‘gates’ above and below the structure, which regulate the number of electrons in the graphene system. When a voltage pulse is applied to the top gate, the initial rise in voltage adds immobile electrons to the top graphene layer. And when the electron energy levels in this layer are filled, mobile electrons are added to the bottom graphene layer. A subsequent decrease in voltage removes electrons from the top graphene layer, however the mobile electrons in the bottom layer remain. In this way, the voltage pulse changes the conductance in a manner that is reminiscent of the strengthening of a synaptic connection.

What sets Yan and co-worker’s moiré device apart from existing synaptic transistors is that it can be easily tuned, a feature that shares similarities with synaptic behaviour observed in biological neural networks. This makes the transistor ideally suited to advanced artificial intelligence (AI) applications, particularly those involving ‘compute-in-memory’ designs that integrate processing circuitry directly into the memory array, to maximize energy efficiency. It could also allow information to be processed on devices located at the edge of a network, rather than in a centralized data centre, thereby enhancing the security and privacy of data.

Although the authors’ transistor represents an important leap forwards, it is not without its limitations. For instance, stacking the ultrathin materials requires sophisticated fabrication processes, which makes it challenging to scale up the technology for widespread industrial use. On a positive note, there are already methods for growing large-area bilayer graphene[4] and hBN[5], up to the typical 200- or 300-millimetre sizes used in the silicon industry. This sets the stage for an ambitious, yet timely, endeavour: the fully automated robotic assembly of large-area moiré materials.

If accomplished, this would make Yan and colleagues’ device easier to fabricate, and unlock other moiré-material innovations, such as quantum sensors, non-volatile computer memories and energy-storage devices. It would also bring us closer to integrating moiré synaptic transistors into larger, more complex neural networks — a crucial step towards realizing the full potential of these devices in real-world applications.

JAMES B. AIMONE & FRANCES S. CHANCE: Capturing the brain’s functionality

Yan and co-workers’ advance addresses a long-standing challenge at the intersection of neuroscience and computing: identifying which biophysical features of the immensely complicated brain are necessary for achieving functional neuromorphic computing, and which can be ignored. The authors have succeeded in emulating a characteristic of the brain that is particularly difficult to realize — its synaptic plasticity, which describes neurons’ ability to control the strength of their synaptic connections.

Existing synaptic transistors can be connected together in grid-like architectures that mimic neural networks. But dynamically reprogramming most of these devices remains unreliable or expensive, whereas the brain’s synapses can adapt reliably and robustly over time. Moreover, even if biological mechanisms of synaptic plasticity can be implemented in an artificial system, it remains unclear how to leverage these mechanisms to realize algorithms that can learn like biological systems do.

The authors’ moiré synaptic transistor brings the flexibility and control necessary for brain-like synaptic learning by providing a powerful way to tune its electrical conductance — a proxy for synaptic strength. The device’s asymmetric charge-transfer mechanism is reminiscent of processes known as long-term potentiation and long-term depression, in which pulsed electrical stimulation has the effect of strengthening a synapse (or weakening it, in the case of depression). The ratcheting of charge carriers can be considered analogous to the enrichment of protein complexes, known as AMPA receptors, at synapses during long-term potentiation and long-term depression[6] (Fig. 1b).

Inspired by observed behaviours in biological synapses, Yan et al. showed that their device could be used to train neuromorphic circuits in a more ‘brain-like’ way than has previously been achieved with artificial synapse devices. Although the two gates in the moiré synaptic transistor could be used in a simple manner to fine-tune synaptic strength (or electrical conductance) directly, in biology, the control of synaptic learning is more nuanced. The authors recognized that aspects of this finer control could also be realized in their device.

Specifically, Yan et al. were able to tune the top and bottom gate voltages to make their moiré synaptic transistor exhibit input-specific adaptation, which is a phenomenon that allows a neuron to control its synaptic learning rates in response to averaged input. This mechanism is used when the eye is deprived (of adequate lighting, for example) to help the brain recall a stored pattern when presented with a similar one.

The authors’ moiré synaptic transistor could emulate this mechanism when programmed with a learning rule known as the Bienenstock–Cooper–Munro (BCM) model[7], which sets a dynamically updated threshold for strengthening or weakening a synapse that depends on the neuron’s history. The BCM rule is an abstract algorithmic description of synaptic plasticity in the brain that has been connected to cognitive behaviours. By demonstrating that their device can implement this rule, Yan et al. have offered a pathway to recreating biorealistic plasticity in human-made hardware.

Their work provides an opportunity for the BCM learning rule to act as a Rosetta Stone between theoretical neuroscience (much of which is based on BCM and similar models) and state-of-the-art neuromorphic computing. For example, the authors’ ingenious dual-gate control could be used to realize synaptic plasticity in the vestibulo-ocular reflex, the mechanism that stabilizes images on the retina as the head moves[8]. It will be interesting to see what other models of plasticity can be expressed, such as spike-timing dependent plasticity, in which the strengthening of a synapse is dependent on the timing of stimulation[9].

Nature 624, 534-536 (2023)

doi: https://doi.org/10.1038/d41586-023-03791-z

References

1. Yan, X. et al. Nature 624, 551–556 (2023).

2. Zheng, Z. et al. Nature 588, 71–76 (2020).

3. Zheng, Z. et al. Preprint at https://arxiv.org/abs/2306.03922 (2023).

4. Huang, M. et al. Nature Nanotechnol. 15, 289–295 (2020).

5. Ma, K. Y. et al. Nature 606, 88–93 (2022).

6. Makino, H. & Malinow, R. Neuron 64, 381–390 (2009).

7. Bienenstock, E. L., Cooper, L. N. & Munro, P. W. J. Neurosci. 2, 32–48 (1982).

8. De Zeeuw, C. I. & Yeo, C. H. Curr. Opin. Neurobiol. 15, 667–674 (2005).

9. Caporale, N. & Dan, Y. Annu. Rev. Neurosci. 31, 25–46 (2008).

Reprints and permissions

Competing Interests

The authors declare no competing interests.

Article Open access Published: 01 November 2023

Homogenizing out-of-plane cation composition in perovskite solar cells

Zheng Liang,

Yong Zhang,

Huifen Xu,

Wenjing Chen,

Boyuan Liu,

Jiyao Zhang,

Hui Zhang,

Zihan Wang,

Dong-Ho Kang,

Jianrong Zeng,

Xingyu Gao,

Qisheng Wang,

Huijie Hu,

Hongmin Zhou,

Xiangbin Cai,

Xingyou Tian,

Peter Reiss,

Baomin Xu,

Thomas Kirchartz,

Zhengguo Xiao,

Songyuan Dai,

Nam-Gyu Park,

Jiajiu Ye &

…

Xu Pan

Show authors

Nature volume 624, pages 557–563 (2023)

- 17k Accesses

- 1 Citations

- 101 Altmetric

- Metrics details

Abstract

Perovskite solar cells with the formula FA1−xCsxPbI_3, where FA is formamidinium, provide an attractive option for integrating high efficiency, durable stability and compatibility with scaled-up fabrication. Despite the incorporation of Cs cations, which could potentially enable a perfect perovskite lattice1,2, the compositional inhomogeneity caused by A-site cation segregation is likely to be detrimental to the photovoltaic performance of the solar cells3,4. Here we visualized the out-of-plane compositional inhomogeneity along the vertical direction across perovskite films and identified the underlying reasons for the inhomogeneity and its potential impact for devices. We devised a strategy using 1-(phenylsulfonyl)pyrrole to homogenize the distribution of cation composition in perovskite films. The resultant p–i–n devices yielded a certified steady-state photon-to-electron conversion efficiency of 25.2% and durable stability.

Similar content being viewed by others

Main

There have been significant improvements in the efficiency of lead-halide perovskite solar cells (PSCs)[5], largely due to the development of new passivation strategies6,7 and the optimization of the perovskite composition[8]. Notably, the modulation of the A-site composition, specifically with FA-Cs alloyed perovskite, where FA is formamidinium, is emerging as a promising method for boosting efficiency[9]. However, there are growing concerns about the stability of Cs-containing perovskites due to the segregation of the cations, which could potentially accelerate the long-term degradation3,4,10,11. The distribution of these inhomogeneous phases within perovskites and their direct impact on efficiency are not yet fully understood.

Herein, we visualize the spatially inhomogeneous phase distribution along the vertical direction across perovskite films and propose that device performance is limited by out-of-plane compositional inhomogeneity. Furthermore, we identified that unbalanced crystallization and phase transition between A-site components have a significant effect on the segregation of the FA and Cs phases. To address this issue, we devised a strategy using 1-(phenylsulfonyl)pyrrole (PSP) as an additive to retard the segregation of cations in FA-Cs perovskites. The PSP-treated devices have a p–i–n structure and yielded a champion photon-to-electron conversion efficiency (PCE) of 26.1% (certified reverse PCE of 25.8% and certified steady-state PCE of 25.2%).

Out-of-plane cation inhomogeneity

The distribution of A-site cations within perovskite films critically affects the performance of the device[12]. Although Cs has been widely used as a cation dopant in perovskite formulations, there are still concerns about the inhomogeneous distribution of cations. Figure 1a is a schematic illustration of the out-of-plane cation inhomogeneity. It shows that Cs prefers to aggregate at the bottom of the perovskite film because crystallization has a significant impact on the compositional evolution within perovskite films. An organic molecule of PSP with a sulfone group13,14 was designed as an additive to the precursor to address the cation inhomogeneity within perovskites, particularly for FA-Cs-containing perovskites (Fig. 1b, Supplementary Fig. 2 and Supplementary Note 1).

Fig. 1: Spatial vertical segregation of the FA and Cs phases.

a, Illustration of inhomogeneous phase distribution caused by out-of-plane FA and Cs segregation. b, Electrostatic potential image and molecular structure of PSP. c, Distribution of cations obtained from ToF-SIMS spectra for the reference (Ref.; blue) and PSP (red) devices. d, Atomic percentage profile of the reference (solid lines) and the PSP (dashed lines) extracted from depth-dependent XPS measurements. e,f, High-angle annular dark-field TEM images for the reference sample (e) and the PSP-treated sample (f). Scale bars, 200 nm. The cross-sectional samples were prepared with a stack configuration of ITO/PTAA/perovskite/PTAA/Cu. The second row underneath each image shows high-resolution TEM images collected from the corresponding boxes. Scale, 7.3 × 7.3 nm. The third row shows the variation of calculated intensity over 3 nm. The calculated interplanar spacing for each lattice is given in the corresponding images. g,h, Enlarged GIXRD spectra collected from the bottom of the reference perovskite film (g) and the PSP-treated perovskite film (h). Structural information with a spatially vertical resolution could be obtained by varying the incident angle of the X-ray beam. In g, the red and blue shading represents the Cs-rich phase and FA-rich phase, respectively. a.u., arbitrary units.

We conducted time-of-flight secondary-ion mass spectroscopy (ToF-SIMS) to investigate the cation distribution (Supplementary Fig. 4). Figure 1c illustrates that in the reference film, for the Cs there is an increasing intensity gradient from the perovskite surface towards the bottom. The FA cations have the opposite trend. This observation confirms the out-of-plane cation inhomogeneity within the perovskite film. Notably, the addition of PSP resulted in a homogeneous cation distribution. Furthermore, as inferred from the characteristic fraction of SO_2−, PSP molecules accumulate at the bottom of the perovskite film. To further survey the elemental variation within the perovskite film, we conducted depth-dependent X-ray photoelectron spectroscopy (XPS). The extracted atomic percentage depth profiles have a similar out-of-plane compositional gradient (Fig. 1d).

The out-of-plane cation inhomogeneity could potentially influence the perovskite lattice, thereby altering the crystal structure. Hence, it is crucial to systematically investigate the structural variations induced by cation inhomogeneities. We visualized the out-of-plane A-site compositional inhomogeneity by studying the lattice heterogeneity of various perovskite phases. We collected cross-sectional transmission electron microscopy (TEM) images. Three regions across the perovskite films (denoted as surface, bulk and bottom) were selected to survey the interplanar spacing of the lattice (d), which is a direct indication of phase heterogeneity. Vertical gradients for the cation distributions were directly observed by comparing the values of d{200}. For the reference film, the three d values were measured to be dsurface = 3.20 Å, dbulk = 3.15 Å and dbottom = 3.11 Å (Fig. 1e). The decreasing trend of the d values correlates with the increasing internal lattice stress within the perovskite film. The decrease in dbottom suggests that there is a significant lattice mismatch at the bottom of the film[15]. This could be ascribed to the relatively smaller Cs atoms accumulating at the bottom, thereby generating a Cs-rich perovskite phase. Importantly, the detected lattice contraction implies that cation inhomogeneity is a contributory factor to the lattice strain16,17. In contrast, as shown in Fig. 1f, negligible variation of the d values was found in the PSP-treated film, which has dsurface = 3.13 Å, dbulk = 3.13 Å and dbottom = 3.13 Å. This finding indicates that PSP treatment provides a better out-of-plane lattice alignment and releases the lattice stress by inhibiting phase segregation.

We employed the grazing incident X-ray diffraction (GIXRD) technique to detect the crystal structure from the exposed lower interface (Fig. 1g and Extended Data Fig. 1). For the reference film, the peaks are at around 28.1°, which are indexed for the (200) plane of perovskite. The split is significantly wider. Shoulder peaks emerge at around 28.4° when the incident angle was lower than 2°. These emergent shoulder peaks gradually weaken with an increase of the incident angle and ultimately vanish. The spectrum changes to stronger integrated peaks when the incident angle is larger than 3°. In contrast, the shoulder peaks are negligible after introducing the PSP (Fig. 1h). Moreover, a vertical misalignment of the peaks, which is an indication of the internal strain caused by lattice mismatch, is observed for the reference and PSP-1.2 films (Supplementary Figs. 5 and  6), which agrees well with the findings of the microscale TEM measurements. These results suggest that there is an out-of-plane inhomogeneous crystal structure within the reference perovskite film. The shoulder peaks may be associated with an undesired Cs-rich phase[2] in the buried region of the perovskite film.

We presume, according to the coherence between 2θ and the lattice space, that the shoulder peaks can be attributed to a Cs-rich phase caused by Cs incorporation15,18,19. Considering the conventional X-ray diffraction (XRD) results (Supplementary Figs. 7 and  8), we may conclude that the spatially out-of-plane compositional inhomogeneity is generated by segregation of the FA and Cs phases in the perovskite films, even for Cs/(Cs + FA) ratios as low as 5%. The Cs-rich phase prefers to accumulate in the bottom region within perovskite films, thus leading to a gradient in the phase distribution from Cs-poor to Cs-rich from the surface to the bottom.

Consequently, the results obtained allow us to conclude that in FA-Cs perovskite films, the different sizes of the cations of FA and Cs result in a spatially out-of-plane lattice mismatch. As shown in Fig. 1a, from top to bottom, there is a FA-rich phase, a phase in which the cations have nominal stoichiometry and a Cs-rich phase.

Origin of the cation inhomogeneity

We performed in situ synchrotron radiation grazing incidence wide-angle X-ray scattering (GIWAXS) to investigate the two critical kinetics processes of crystallization and phase transition during perovskite formation. As demonstrated in Fig. 2a, signals for a q vector of around 0.8, 0.82 and 1.0 Å−1 can be assigned to the δ-phase perovskite of 2H (100), 6H (101), and α-phase perovskite, respectively[20]. We defined two periods when analysing the kinetic processes. Period I was from after chlorobenzene dripping until the emergence of α-phase perovskite. The duration of this period is indicative of the crystallization rate. Period II was the duration it took for the α phase to become stable, which reflects the δ- to α-phase transition rate. From the in situ GIWAXS results, we found that the introduction of PSP accelerates both the crystallization and phase transition. Combined with the results from ‘Out-of-plane cation inhomogeneity’, this shows that PSP has effectively inhibited the segregation of the FA and Cs phases. A possible kinetical culprit for the phase segregation is the slow speeds of crystallization and phase transition.

Fig. 2: Revealing the origin of the segregation of the FA and Cs phases.

a, In situ GIWAXS pattern revealing processes of crystallization and phase transition. The  colour bars range from 0 to 1. b,c, Schematics of computation results for free energy evolution in the reference system (b) and PSP system (c) during crystallization and the phase transition. The blue and red rectangles represent the relevant FAPbI_3 phases and CsPbI_3 phases, respectively. The blue and red solid lines indicate the free energy evolution of FAPbI_3 and CsPbI_3, respectively. d,e, Plots of Fourier-transformed R space results of EXAFS measurements of the reference films (d) and PSP films (e). The dashed lines at 2.9 Å and 2.2 Å correspond to the Pb–I and Pb–O coordination, respectively. f, Pb–O coordination ratios calculated from the EXAFS measurements. g, FTIR spectra of PSP and the PSP(PbI_2) complex. CB, chlorobenzene.

We further conducted density functional theory computations to thermodynamically investigate the barrier energy (EB) for perovskite crystallization and phase transition and subdivided the energy into the processes for the FA and Cs components. The difference between the barrier energies was defined as \(\Delta {E}_{{\rm{B}}}={E}_{{\rm{B}}}^{{\rm{FA}}}-{E}_{{\rm{B}}}^{{\rm{Cs}}}\). To evaluate more accurately the imbalance between the FA and Cs components, we calculated the mismatch factor \(\mu =\frac{{E}_{{\rm{B}}}^{{\rm{FA}}}-{E}_{{\rm{B}}}^{{\rm{Cs}}}}{{E}_{{\rm{B}}}^{{\rm{FA}}}}\). As shown in Fig. 2b,c, during period I, for the reference system, ΔEB,Iref = 101.6 meV. After the PSP was introduced, ΔEB,IPSP = 35.3 meV. The corresponding μ values were calculated to be μI,ref = 20.48% and μI,PSP = 5.34%. During period II, for the reference system, ΔEB,IIref = 82 meV with μref = 12.49%, whereas ΔEB,IIPSP = −6 meV with μPSP = −1.79% for the PSP system (Supplementary Table 2). The lower μI and μII for the PSP system indicate that the differences in the crystallization and phase transition rates of the FA and Cs components were reduced. Such differences in the rates for cations were probably responsible for the tardiness observed by in situ GIWAXS. A possible reason for the cation inhomogeneity could be the soft base property of the Cs cations compared to the FA cations, which may lead to a much more intensive interaction with PbI_3−, leading to Cs preferentially aggregating at the bottom. Additionally, the difference in the solubilities of Cs and FA components might partially also contribute to the cation inhomogeneity[21].

We collected adsorption spectra of the Pb LIII edge using extended X-ray absorption fine spectroscopy (EXAFS) to evaluate the interactions between PSP and perovskite. We selected five grazing incident angles to capture information at various depths within the perovskite film (Supplementary Fig. 12). As shown in Fig. 2d,e, peaks at radial distances of approximately 2.2 and 2.9 Å can be attributed to Pb–O and Pb–I coordination, respectively[22]. In the reference film, we observed a gradual downwards shift of around 0.03 Å for the Pb–I coordination with an increase in detection depth (Fig. 2d). This indicates that the lattice was compressed at the bottom of the perovskite[23]. In contrast, the peaks associated with Pb–I coordination remained relatively stable upon the incorporation of PSP, further reinforcing the existence of out-of-plane cation inhomogeneity. Notably, in the PSP film, peaks corresponding to Pb–O coordination migrated by a higher radial distance as the depth increased (Fig. 2e). This suggests the creation of a longer Pb–O coordination at the bottom of the film. By calculating of the Pb–O coordination ratio as (Pb–O)/((Pb–O) + (Pb–I)) (Fig. 2f), we concluded that Pb atoms at the bottom of the perovskite film tend to coordinate with additional oxygen atoms from PSP. Consequently, we hypothesize that PSP possibly interacts with the Pb atoms in perovskite through electrons donated by its two oxygen atoms.

To precisely evaluate the interaction between PSP and PbI_2, we synthesized (PbI_2)x(PSP)y complex crystals (Extended Data Fig. 2 and Supplementary Fig. 13), and performed Fourier transform infrared spectroscopy (FTIR) measurements. Peaks at around 1,328, 1,133 and 964 cm−1 correspond to asymmetric stretching vibration (νas) and symmetric stretching vibration (νs) of the sulfone (O=S=O) and vibration (ν) of the sulfoxide (S=O) group, respectively. Significant shifts in all three characteristic peaks indicate the coordination of PSP with PbI_2 through the sulfone (O=S=O) group (Fig. 2g). The upward shifts of the νas and νs peaks imply that both oxygen atoms from PSP may serve as active sites. Additionally, nuclear magnetic resonance (NMR) spectra corroborated the coordination of the O=S=O group with PbI_2. This was discerned through shifts in the carbon atoms adjacent to the O=S=O group (nos. 1, 2, 6, 11 and 14) to a higher field (Extended Data Fig. 3). These findings align with the decreased PbI_2 signal observed by in situ GIWAXS tests and the peak shifts detected in XPS measurements (Supplementary Fig. 14).

Optoelectronic properties

Besides retarding the phase segregation, PSP has a practical passivation effect. The steady-state and time-resolved photoluminescence were examined to optically evaluate changes in recombination caused by the out-of-plane lattice mismatch. In Fig. 3a, a significantly stronger photoluminescence peak was observed for the PSP film relative to that of the reference film. Moreover, the carrier lifetime of the PSP-treated film was extended to 1,876.6 ns compared with 491.7 ns for the reference (Fig. 3b and Supplementary Table 3). We used thermal admittance spectroscopy to characterize the trap density of the perovskite films. Supplementary Fig. 17 depicts that the trap density of states decreased after PSP introduction at shallow and deep energetic levels. Shallow traps could be attributed to a homogeneous phase distribution, which may inhibit the formation of vacancies. Further, a released spatial lattice mismatch is beneficial for stabilizing octahedral frameworks, which in turn positively reduces the metal-related deep defects24,25. We further modelled the lattice and calculated the defect formation energy using FA0.95Cs0.05PbI_3 perovskite (Fig. 3c and Supplementary Fig. 19). Figure 3d shows that the defect formation energy of a series of defects was increased after PSP introduction, especially for the Pb and I vacancies, which should result in lower defect densities and longer carrier lifetimes in experiments.

Fig. 3: Optoelectronic properties.

a, Steady-state photoluminescence (PL) spectra of perovskite films with and without PSP treatment deposited onto quartz glass substrates. b, Time-resolved PL spectra of perovskite films deposited onto quartz glass substrates. The solid lines are fitted using the dual exponential fitting method. c, Constructed lattice model of FA0.95Cs0.05PbI_3 perovskite. The (100) plane of the lattice was exposed so that it could adsorb a PSP molecule for the density functional theory computation. d, Statistics for the defect formation energy for various types of defect in the reference and PSP systems. e,f, Schematics for the band alignment within the reference perovskite film (e) and the PSP perovskite film (f). The values of the conduction-band minimum, valence-band maximum and Fermi level (EF) were extracted from the depth-profile ultraviolet-photoelectron spectrum. The schematics were finally obtained by combining data from three depth regions with a manually aligned EF. HTL, hole-transport layer; ETL, electron-transport layer.

Efficient carrier diffusion and extraction are affected by the vertical band alignment of different phases within a perovskite film, which correlates with the out-of-plane compositional inhomogeneity. Depth-profile ultraviolet-photoelectron spectroscopy was carried out to evaluate the internal band alignment (Extended Data Fig. 4). Figure 3e shows that the out-of-plane compositional inhomogeneity may lead to quasi-type I band alignment at the contact region of the Cs-rich phase with a thickness of a few hundreds of nanometres. The conduction-band minimum and valence-band maximum are downwards and upwards twisted, respectively. This band alignment adversely affects carrier transport for solar cells through electrical doping[26] whether in a p–i–n or n–i–p configuration (Extended Data Fig. 5). In this case, the inherent disequilibrium of electron–hole extraction would be seriously aggravated27,28. Ultimately, it would worsen the device efficiency, especially the fill factor29,30,31. After PSP introduction, the band diagram had a favourable flattened alignment, mitigating energy losses of charge carriers within perovskite films (Fig. 3f). The findings from transient adsorption measurements align with the conclusion obtained from energy band alignment (Extended Data Fig. 6). Moreover, we tested the built-in electrical field (Vbi) in a p–i–n device. The improved Vbi was beneficial for carrier diffusion as well (Supplementary Fig. 20).

Device performance

We fabricated devices with a p–i–n stack of indium tin oxide (ITO)/poly(triaryl)amine (PTAA)/FA0.95Cs0.05PbI_3/C_60/bathocuproine (BCP)/Ag. The bandgap of this perovskite recipe was determined to be 1.51 eV by the Tauc plot method (Supplementary Figs. 21) and by using the derivative of the external quantum efficiency (EQE) of the solar cell (Extended Data Fig. 7). The champion device yielded a notable PCE of up to 26.09%/25.16% (reverse/forward scan direction) whereas the reference cell had PCEs of 24.62%/23.48% (Fig. 4a). The corresponding steady-state power output efficiencies were 25.15% and 23.72%, respectively. An unencapsulated device achieved certified PCEs of 25.8% and 25.2% for the reverse scan and for the steady-state output, as certified by an independent organization (Supplementary Fig. 22). The fill factor of the champion device exceeded 85%, which is nearly 95% of the theoretical limit (89.5%). We attributed the remarkable fill factor improvement to the improved charge carrier extraction, which is supported by the efficiency improvement in the n–i–p configuration as well (Supplementary Fig. 23). The open-circuit voltage (VOC) improved from 1.145 to 1.164 V, which is consistent with the reduced trap density. The short-circuit current density (JSC) over 26 mA cm−2 was consistent with the integrated JSC extracted from the incident photon-to-electron conversion efficiency (IPCE) (Fig. 4b). The reproducibility was evaluated using a batch of devices comprising 16 individuals for each set-up (Supplementary Fig. 25). We further tested the solar cells in a light-emitting diode (LED) mode. The LED EQEEL improved from 7.1% to 9.7% (Fig. 4c), so that the emission peaks were around 820 nm (Supplementary Fig. 26). We fabricated devices with an upscaled area of 1 cm^2, and the efficiency improved from 21.78% to 23.64% (Fig. 4d). This improvement was mainly associated with the enhanced fill factor. Further devices with typical efficient perovskite formulas were fabricated to assess the universality of the PSP strategy (Supplementary Figs. 27–29), although some perovskite formulas still require further evaluation.

Fig. 4: Device performance and stability.

a, J–V curves of champion p–i–n PSCs at laboratory scale. The active area was around 0.073 cm^2. The inset shows the detailed photovoltaic parameters from the reverse scan and the steady-state power output. b, IPCE plots for the PSP solar cells. The solid red line is the integrated JSC. c, EQE curves measured for the reference and PSP solar cells in LED mode. The inset is a photograph showing the PSCs working in LED mode. d, J–V curves of scaled-up PSCs with and without PSP. The inset is a photograph showing the scaled-up PSCs with 1 cm^2 active area. e, Normalized evolution of the PCE for unencapsulated reference and PSP devices under continuous tracking at the maximum power point following the ISOS L-1I protocol. The initial PCEs of the reference and PSP devices were 25.40% and 23.54%, respectively. RT, room temperature. f, Results of damp-heat reliability tests of the encapsulated devices tested at 85 °C and 85% relative humidity (RH) following the ISOS D-3 protocol. A device stack based on ITO/PTAA/perovskite/C_60/Au was used in the damp-heat tests. The solid lines represent the average PCE for six individual devices. The initial average PCEs of the reference and PSP devices were 19.2% and 21.8%. The error bars denote the standard deviation. champ., champion; FF, fill factor.

Device reliability was evaluated following the procedure specified in the ISOS protocols[32]. An unencapsulated PSP-treated device retained 92% of its initial PCE after 2,500 h of continuous tracking at the maximum power point in a nitrogen atmosphere. By contrast, the PCE of the reference device dropped to around 80% of its initial value under the same conditions (Fig. 4e). Damp-heat experiments were conducted using encapsulated devices in an ageing box with 85 °C and 85% relative humidity. The PSP-treated device exhibited almost 90% efficiency after over 300 h on average, compared with around 80% for the initial PCE of the reference (Fig. 4f). Regarding the temperature cycling reliability, the encapsulated PSP-treated device retained 93% of its initial PCE after 300 cycles, compared to 67% for the reference (Extended Data Fig. 8). Overall, these findings provide an in-depth understanding of phase segregation and suggest a promising strategy for accelerating commercialization of perovskite photovoltaics.

Methods

Materials

Lead iodide (PbI_2, 99.999%), lead bromide (PbBr_2, 99.999%), caesium iodide (CsI, 99.999%), bis(trifluoromethane)sulfonimide lithium salt (Li-TFSI, 99.95%) and 4-tert-butylpyridine were purchased from Sigma Aldrich. Formamidinium iodide, methylammonium (MA) bromide and methylammonium chloride (MACl) were synthesized in house by reacting equal molar amounts of formamidine (FA) acetate and methylamine alcohol solution with the corresponding halogen acid[33]. PSP was synthesized in a laboratory according to the method mentioned in the Supplementary Information. PTAA, 2,2′,7,7′-tetrakis[N,N-di(4-methoxyphenyl)amino]-9,9′-spirobifluorene (spiro-OMeTAD) and BCP were purchased from Lumtec. Tin (IV) oxide (SnO_2, 15% in H_2O colloidal dispersion) was purchased from Alfa Aesar. Fullerene (C_60) was purchased from Nano-C. All solvents used in the experiments, including N,N-dimethylformamide (99.8%), dimethyl sulfoxide (anhydrous, 99.9%), chlorobenzene (anhydrous, 99.8%), acetonitrile (99.9%) and isopropyl alcohol (anhydrous, 99.8%), were purchased from Sigma Aldrich. All chemicals were used as received without any further purification.

Perovskite precursor solution

The perovskite precursor solution was prepared by dissolving FAPbI_3 (1.71 M) powder and CsPbI_3 powder (0.09 M) in a mixture of the solvents dimethylformamide and dimethyl sulfoxide (8:1 v/v). 0.5% MAPbBr_3 powder, 7.2% PbI_2 and 30% MACl with molar ratio were added to increase the crystallinity[34]. For a solution with PSP agents, PSP powder was directly added into the perovskite precursor solution with different concentrations. PSP at 1.2, 2.4 and 4.8 mg ml−1 was added to perovskite precursors for comparison (denoted as PSP-1.2, PSP-2.4 and PSP-4.8, respectively). The obtained precursor was vigorously shaken at room temperature for over 3 h. Finally, the perovskite precursor solutions were filtered through 0.22 μm polytetrafluoroethylene filters before use.

Device fabrication

The devices with the p–i–n configuration were fabricated as follows. Prepatterned ITO-coated glass substrates (7 Ω per square) were cleaned by sequential ultrasonication in detergent, deionized water, acetone and isopropanol each for 15 min, respectively. The cleaned substrates were dried under a flow of clean N_2 and further dried in a 60 °C oven overnight. Before depositing the hole transport layer, the substrates were exposed to oxygen plasma cleaner for 5 min and then transferred into an N_2 glovebox immediately for rest deposition progress. PTAA solution (2.5 mg ml−1 dissolved in chlorobenzene) was spin-coated onto the substrate at 6,000 rpm for 30 s, then annealed at 100 °C for 15 min. The perovskite layer was deposited by one-step spin-coating of the filtered precursor solution at 1,000 rpm for 10 s and 4,000 rpm for 40 s. Then 200 μl of chlorobenzene was quickly dropped onto the centre of the spinning substrate at 15 s before the end. The film was immediately annealed at 100 °C for 30 min. After cooling down to room temperature, 100 μl of phenethylammonium iodide or n-octylammonium iodide solution (5 mg ml−1) was dynamically coated onto the perovskite films at 3,000 rpm for 30 s, followed by annealing at 100 °C for 5 min. The device fabrication was accomplished after sequential thermal evaporation of C_60 (30 nm, 0.1 Å s−1), BCP (7 nm, 0.1 Å s−1) and Ag (100 nm, 0.2 Å s−1) in a high-vacuum chamber (7 × 10−5 Pa). Note that all procedures for device fabrication were conducted in a nitrogen glovebox (O_2 and H_2O less than 0.1 ppm).

The devices with the n–i–p configuration were fabricated as follows. After the same pretreatment of the ITO substrates, SnO_2 in a colloidal dispersion (diluted by 1:4 v/v with deionized H_2O) was spin-coated onto the substrates at 3,000 rpm for 30 s, and sequentially annealed at 180 °C for 50 min in air (35% relative humidity). The procedures for perovskite film deposition were the same as those for the p–i–n devices. A solution of hole transport material was prepared 30 min before use by mixing spiro-OMeTAD (91 mg) and 4-tert-butylpyridine (36 μl) in chlorobenzene (1 ml). Li-TFSI (23 μl, 520 mg ml−1 in acetonitrile) was doped to improve its conductivity. 50 μl of the solution of hole transport material was dynamically spin-coated onto the perovskite films at 3,000 rpm for 30 s in a nitrogen-filled glovebox. Finally, gold electrodes (100 nm, 0.2 Å s−1) were deposited through thermal evaporation.

Photovoltaic performance characterization

The J–V measurements were carried out with a xenon lamp-based solar simulator (Enlitech SS-F5-3A, Class AAA) and a source meter (Keithley 2400). The simulated AM 1.5G irradiation (100 mW cm−2) was calibrated by a standard silicon cell (traced to NREL, SRC-2020). The solar cells were measured with a metal mask with an area of 7.485 mm^2 to accurately define the active area. The voltage was applied from −0.2 to 1.3 V with a scanning rate of 0.2 V s−1, and the voltage step was 20 mV. All devices were measured immediately after fabrication in an N_2 glovebox. The IPCE was measured in a.c. mode on the xenon lamp-based system (Newport TLS260-300X). The scan range was from 300 to 1,000 nm. The solar cells were measured in LED mode in N_2 using a home-made motorized goniometer set-up consisting of a source meter unit (Keithley 2400), a calibrated Si photodiode (FDS-100-CAL, Thorlabs), a pico-ammeter (4140B, Agilent) and a calibrated fibre optic spectrophotometer (UVN-SR, StellarNet Inc.). The distance between the LED device and the photodetector was 59.5 mm.

Stability characterization

A home-made calibrated LED-based solar simulator with an intensity of 100 mW cm−2 was used as an illumination source in the stability tests. The tracking of the maximum power point was performed in N2. A blower was used to ensure the device was at a constant temperature. A home-made system was used to acquire the continuous power evolution. Devices for the damp-heat test were encapsulated with epoxy. The Ag electrodes were replaced by Au, and BCP was removed from the device. The damp-heat tests and thermal cycling tests were conducted in a customized ageing box. The accuracies of the temperature and humidity were under ±1 °C and ±5%, respectively. The damp-heat tests were periodically performed during the J–V scan after cooling down in N_2 for around 30 min. The thermal cycling tests were carried out by repeatedly performing the J–V characterization after the stages for stabilizing the temperature.

Cross-sectional microstructure characterization

Note that all measurements, unless otherwise specified, were conducted with a perovskite formulation of FA0.95Cs0.05PbI_3. The samples were prepared with ITO/PTAA/perovskite/PTAA/Cu stacks, in which a higher concentration of PTAA solution (30 mg ml−1 in chlorobenzene) and thermally evaporated copper (200 nm, 0.2 Å s−1) were used. Thick layers of PTAA and copper on the perovskite films can protect the samples from milling damage[35]. The other procedures were the same as in device fabrication apart from the absence of surface passivation. Pt and carbon layers were deposited before thinning using focused ion beam (FIB) equipment (ThermoFisher Helios 5 CX). A thick plate was extracted from the bulk sample at 30 kV and 3,000–30,000 pA, which was welded onto a Cu grid (omniprobe grid) by a probing system. The thick plate was first thinned at 30 kV and 50–1,000 pA and then at a lower current of 10–30 pA. Finally, the specimen was completed by thinning at a lower voltage of 1 kV and 30–50 pA after beam showering at 3 kV and 10–30 pA. The samples were immediately transferred to a TEM system (ThermoFisher Talos F200S). High-angle annular dark-field images across the full cross section of the samples and high-resolution TEM images were collected at an acceleration voltage of 200 kV.

Structure characterization

XRD and GIXRD data were acquired from a diffractometer (SmartLab, Rigaku) using Cu Kα (λ = 1.5406 Å) radiation. The tests were performed by scanning 2θ of 5°–45° with a scan rate of 3° min−1 and 0.02° per step. The GIXRD tests detected signals from the bottom side of the perovskite films at grazing incident angles of 0.1°, 0.4°, 0.8°, 1°, 2°, 3°, 4° and 5°. The corresponding penetration depths for the perovskite material were calculated[36] as described in the Supplementary Information.

Depth profiling characterization

The depth profiles of perovskite deposited onto ITO substrates were recorded using a ToF-SIMS system (TOF-SIMS 5, ION-TOF) with a Bi3+ primary beam (25 keV, 1 pA) and an oxygen sputter beam (1 keV, 45 nA). Note that an oxygen sputter gun can help in expelling pollution from the perovskite surface[37]. Samples were prepared with the full solar cell configuration for the depth-dependent photoelectron spectroscopy with an integrated etching system (ThermoFisher, ESCALAB Xi+).

Synchrotron radiation characterization

Pb LIII-edge EXAFS data were collected on the BL13SSW beamline at the Shanghai Synchrotron Radiation Facility (SSRF) using the top-up mode operation with a ring current of 200 mA at 3.5 GeV. From the high-intensity X-ray photons of the multipole wiggler source, monochromatic X-ray beams could be obtained using a liquid-nitrogen-cooled double-crystal monochromator with a Si(111) crystal pair. For each grazing incident angle, X-ray absorption spectra were recorded in fluorescence mode using an N_2/Ar mixed-gas-filled ionization chamber and passivated implanted planar silicon (Canberra Co.) for the incident and fluorescent X-ray photons, respectively. Higher-order harmonic contamination was eliminated by detuning to reduce the incident X-ray intensity by about 30%. The energy calibration was performed with a Pb foil reference using the Athena package[38]. Fourier-transformed radial distribution functions of k3-weighted Pb LIII-edge EXAFS spectra k^3χ(k) were obtained in the k range between 3.0 and 9.0 Å−1 through a standard XAFS data-analysis process. In situ GIWAXS tests were performed at the beamlines BL14B1 and BL17B1 of SSRF. A two-dimensional detector (Rayonix MX300) was used to capture 360-frame spectra with 2 s intervals during spin-coating. Chlorobenzene dripping was automatically controlled in this experiment.

Other characterizations

The ultraviolet to visible absorption spectra were acquired with a spectrophotometer (Lambda 365, PerkinElmer). The photoluminescence and time-resolved photoluminescence were measured with a spectrofluorometer (Horiba Fluorolog-3 system). The excitation wavelength for the photoluminescence was 480 nm. The time-resolved photoluminescence was measured using a 532 nm laser nano-LED as an excitation source. All samples were deposited onto quartz glass. The morphology images were collected by a scanning electron microscope (Gemini SEM 500, Zeiss). The Mott–Schottky plots were measured with an applied bias range from −0.1 to 1.2 V. The built-in potential was determined using the equation \({\left(\frac{A}{C}\right)}^{2}=\frac{2}{q{\varepsilon }_{{\rm{r}}}{\varepsilon }_{0}{N}_{{\rm{D}}}}({V}_{{\rm{bi}}}-V)\) where A is the device area, C the capacitance, q the elementary charge, Vbi the built-in potential, V the applied voltage, εr the dielectric constant, ε_0 the permittivity of free space and ND the carrier density. The depletion width was calculated using \(W={\left(\frac{2{\varepsilon }_{{\rm{r}}}{\varepsilon }_{0}}{{N}_{{\rm{D}}}}{V}_{{\rm{bi}}}\right)}^{1/2}\). The capacitance–frequency curves were measured over a frequency range from 10^1 to 10^6 Hz using an electrochemical workstation (Zahner IM6ex). An a.c. amplitude voltage of 5 mV was used, and the d.c. bias was kept at 0 V to avoid the influence of the ferroelectric effect. The final trap density of states was calculated using \({\rm{tDOS}}({E}_{\omega })=-\frac{{\rm{d}}C}{{\rm{d}}\omega }\frac{{V}_{{\rm{bi}}}}{qW}\frac{\omega }{{k}_{{\rm{B}}}T}\) where ω is the angular frequency, W the depletion width, kB the Boltzmann constant and T temperature. Here, \({E}_{\omega }={k}_{{\rm{B}}}Tln\left(\frac{{\omega }_{0}}{\omega }\right)\) where Ω_0 is the attempt-to-escape frequency at temperature T.

Computational details

All the spin theoretical simulations in our work were carried out with the Vienna ab initio Simulation Package (VASP) v.5.4.4. The generalized gradient approximation with the Perdew–Burke–Emzerhof functional form was employed to evaluate the electron–electron exchange and correlation interactions. Projector augmented-wave methods were implanted to represent the core-electron (valence electron) interactions. The plane-wave basis function was set with a kinetic cutoff energy of 550 eV. The ground-state atomic geometries were optimized by relaxing the force below 0.02 eV/Å and the convergence criteria for energy was set with a value of 1.0 × 10−5 eV per cell. The Brillouin zone was sampled using Monkhorst–Pack meshes of size 5 × 5 × 1 for the slab models. All slab models were modelled with a 20 Å vacuum layer. Gaussian smearing was employed for the stress/force relaxations. To better describe the interactions between molecules, van der Waal interactions were included with the zero damping DFT-D3 method of Grimme. The transition states during the reaction pathway were evaluated with the climbing-image nudged elastic band method. The convergence criteria for force were below 0.05 eV/Å. Only the gamma point was considered in this calculation.

Data availability

The data that support the findings of this study are available from the corresponding author (X.P.) upon request.

References

1. Li, Z. et al. Stabilizing perovskite structures by tuning tolerance factor: formation of formamidinium and cesium lead iodide solid-state alloys. Chem. Mater. 28, 284–292 (2015).

2. Yi, C. et al. Entropic stabilization of mixed A-cation ABX3 metal halide perovskites for high performance perovskite solar cells. Energy Environ. Sci. 9, 656–662 (2016).

3. Li, N. et al. Microscopic degradation in formamidinium-cesium lead iodide perovskite solar cells under operational stressors. Joule 4, 1743–1758 (2020).

4. Bai, Y. et al. Initializing film homogeneity to retard phase segregation for stable perovskite solar cells. Science 378, eabn3148 (2022).

5. Yoo, J. J. et al. Efficient perovskite solar cells via improved carrier management. Nature 590, 587–593 (2021).

6. Jiang, Q. et al. Surface reaction for efficient and stable inverted perovskite solar cells. Nature 611, 278–283 (2022).

7. Zhao, Y. et al. Inactive (PbI_2)2RbCl stabilizes perovskite films for efficient solar cells. Science 377, 531–534 (2022).

8. Kim, G. et al. Impact of strain relaxation on performance of α-formamidinium lead iodide perovskite solar cells. Science 370, 108–112 (2020).

9. Bu, T. et al. Lead halide-templated crystallization of methylamine-free perovskite for efficient photovoltaic modules. Science 372, eabh1035 (2021).

10. Liu, L. et al. A-site phase segregation in mixed cation perovskite. Mater. Rep., Energy 1, 100064 (2021).

11. Correa-Baena, J.-P. et al. Homogenized halides and alkali cation segregation in alloyed organic–inorganic perovskites. Science 363, eaah5065 (2019).

12. Lee, J. W. et al. Rethinking the A cation in halide perovskites. Science 375, eabj1186 (2022).

13. Jeon, N. J. et al. Solvent engineering for high-performance inorganic–organic hybrid perovskite solar cells. Nat. Mater. 13, 897–903 (2014).

14. Ren, Y. et al. A bi-functional additive for linking PI_2 and decreasing defects in organo-halide perovskites. J. Alloys Compd. 758, 171–176 (2018).

15. Kim, H.-S. & Park, N.-G. Importance of tailoring lattice strain in halide perovskite crystals. NPG Asia Mater. 12, 78 (2020).

16. Turren-Cruz, S.-H. et al. Methylammonium-free high-performance, and stable perovskite solar cells on a planar architecture. Science 362, eaat3583 (2018).

17. Zhu, C. et al. Strain engineering in perovskite solar cells and its impacts on carrier dynamics. Nat. Commun. 10, 815 (2019).

18. Kubicki, D. J. et al. Phase segregation in Cs-, Rb- and K-doped mixed-cation (MA)x(FA)1−xPbI_3 hybrid perovskites from solid-state NMR. J. Am. Chem. Soc. 139, 14173–14180 (2017).

19. Chen, L. et al. Toward long-term stability: single-crystal alloys of cesium-containing mixed cation and mixed halide perovskite. J. Am. Chem. Soc. 141, 1665–1671 (2019).

20. Proppe, A. H. et al. Multication perovskite 2D/3D interfaces form via progressive dimensional reduction. Nat. Commun. 12, 3472 (2021).

21. Zheng, Y. et al. Downward homogenized crystallization for inverted wide‐bandgap mixed‐halide perovskite solar cells with 21% efficiency and suppressed photo‐induced halide segregation. Adv. Funct. Mater. 32, 2200431 (2022).

22. Kang, D.-H. et al. Extended X-ray absorption fine structure (EXAFS) of FAPbI_3 for understanding local structure–stability relation in perovskite solar cells. J. Energy Chem. 67, 549–554 (2022).

23. Park, J. et al. Controlled growth of perovskite layers with volatile alkylammonium chlorides. Nature 616, 724–730 (2023).

24. Zhou, Y., Poli, I., Meggiolaro, D., De Angelis, F. & Petrozza, A. Defect activity in metal halide perovskites with wide and narrow bandgap. Nat. Rev. Mater. 6, 986–1002 (2021).

25. Shao, Y. et al. Origin and elimination of photocurrent hysteresis by fullerene passivation in CH_3NH_3PbI_3 planar heterojunction solar cells. Nat. Commun. 5, 5784 (2014).

26. Euvrard, J. et al. Electrical doping in halide perovskites. Nat. Rev. Mater. 6, 531–549 (2021).

27. Eperon, G. E. et al. Formamidinium lead trihalide: a broadly tunable perovskite for efficient planar heterojunction solar cells. Energy Environ. Sci 7, 982–988 (2014).

28. Herz, L. M. Charge-carrier mobilities in metal halide perovskites: fundamental mechanisms and limits. ACS Energy Lett. 2, 1539–1548 (2017).

29. Peng, J. et al. Centimetre-scale perovskite solar cells with fill factors of more than 86 per cent. Nature 601, 573–578 (2022).

30. Ding, Y. et al. Single-crystalline TiO_2 nanoparticles for stable and efficient perovskite modules. Nat. Nanotechnol. 17, 598–605 (2022).

31. Cao, Q. et al. Efficient and stable inverted perovskite solar cells with very high fill factors via incorporation of star-shaped polymer. Sci. Adv. 7, eabg0633 (2021).

32. Khenkin, M. V. et al. Consensus statement for stability assessment and reporting for perovskite photovoltaics based on ISOS procedures. Nat. Energy 5, 35–49 (2020).

33. Jeong, J. et al. Pseudo-halide anion engineering for α-FAPbI_3 perovskite solar cells. Nature https://doi.org/10.1038/s41586-021-03406-5 (2021).

34. Min, H. et al. Perovskite solar cells with atomically coherent interlayers on SnO_2 electrodes. Nature 598, 444–450 (2021).

35. Zhang, Y. et al. Achieving reproducible and high-efficiency (>21%) perovskite solar cells with a presynthesized FAPbI_3 powder. ACS Energy Lett. 5, 360–366 (2019).

36. Chateigner, D. Book review of Thin Film Analysis by X-ray Scattering by M. Birkholz with contributions by P. F. Fewster and C. Genzel. J. Appl. Crystallogr. 39, 925–926 (2006).

37. Harvey, S. P. et al. Probing perovskite inhomogeneity beyond the surface: TOF-SIMS analysis of halide perovskite photovoltaic devices. ACS Appl. Mater. Interfaces 10, 28541–28552 (2018).

38. Rehr, J. J. et al. Theoretical X-ray absorption fine structure standards. J. Am. Chem. Soc. https://doi.org/10.1021/ja00014a001 (1991).

Acknowledgements

This work was financially supported by the National Key R&D Program of China (grant no. 2021YFB3800102), the National Natural Science Foundation of China (grant nos. 52272252, U22A20142, 52302324 and 62204108), the Director’s Fund of the Hefei Institutes of Physical Science (grant no. YZJJ-GGZX-2022-01) and the Science Funds of Distinguished Young Scholars of Anhui Province (grant no. 2108085J34). We acknowledge use of the beamlines BL13SSW, BL14B1, BL17B1 and BL10U2 at SSRF, China, for the synchrotron radiation experiments. We thank S. He, Z. Su and L. Cheng from SSRF for support with the synchrotron radiation experiments. N.-G.P. acknowledges financial support through grants from the National Research Foundation of Korea, which is funded by the Korean Ministry of Science and ICT under contract NRF-2021R1A3B1076723 (Research Leader Program). S.D. acknowledges support from the 111 project (B16016) and the Beijing Key Laboratory of Novel Thin-Film Solar Cells.

Author information

Author notes

1. These authors contributed equally: Zheng Liang, Yong Zhang, Huifen Xu

Authors and Affiliations

1. Key Laboratory of Photovoltaic and Energy Conservation Material, Institute of Solid-State Physics (ISSP), Hefei Institutes of Physical Science (HIPS), Chinese Academy of Sciences, Hefei, People’s Republic of China

Zheng Liang, Huifen Xu, Boyuan Liu, Hui Zhang, Zihan Wang, Huijie Hu, Xingyou Tian, Jiajiu Ye & Xu Pan

2. University of Science and Technology of China (USTC), Hefei, People’s Republic of China

Zheng Liang, Huifen Xu, Wenjing Chen, Boyuan Liu, Hui Zhang, Zihan Wang, Huijie Hu, Hongmin Zhou & Zhengguo Xiao

3. Shenzhen Engineering Research and Development Center for Flexible Solar Cells, Southern University of Science and Technology (SUSTech), Shenzhen, People’s Republic of China

Yong Zhang, Jiyao Zhang & Baomin Xu

4. Department of Materials Science and Engineering, Southern University of Science and Technology (SUSTech), Shenzhen, People’s Republic of China

Yong Zhang, Jiyao Zhang & Baomin Xu

5. Hefei National Laboratory for Physical Sciences at the Microscale and Department of Physics, University of Science and Technology of China (USTC), Hefei, People’s Republic of China

Wenjing Chen & Zhengguo Xiao

6. School of Chemical Engineering and Center for Antibonding Regulated Crystals, Sungkyunkwan University (SKKU), Suwon, Republic of Korea

Dong-Ho Kang & Nam-Gyu Park

7. SKKU Institute of Energy Science & Technology (SIEST), Sungkyunkwan University, Suwon, Republic of Korea

Dong-Ho Kang & Nam-Gyu Park

8. Shanghai Synchrotron Radiation Facility (SSRF), Shanghai Advanced Research Institute (SARI), and Shanghai Institute of Applied Physics, Chinese Academy of Sciences (CAS), Shanghai, People’s Republic of China

Jianrong Zeng, Xingyu Gao & Qisheng Wang

9. Instruments Center for Physical Science (PIC), University of Science and Technology of China (USTC), Hefei, People’s Republic of China

Hongmin Zhou

10. Department of Physics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, People’s Republic of China

Xiangbin Cai

11. University Grenoble-Alpes, CEA, CNRS, INP, IRIG/SyMMES, STEP, Grenoble, France

Peter Reiss

12. IEK5-Photovoltaics, Forschungszentrum Jülich, Jülich, Germany

Thomas Kirchartz & Jiajiu Ye

13. Faculty of Engineering and CENIDE, University of Duisburg-Essen, Duisburg, Germany

Thomas Kirchartz

14. State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources, North China Electric Power University (NCEPU), Beijing, People’s Republic of China

Songyuan Dai

Contributions

Z.L., J.Y. and X.P. conceived the idea. J.Y. and X.P. supervised the project and process. Z.L. and H.X. fabricated the perovskite devices, conducted the photovoltaic performance characterizations and analysed the experimental data described in this manuscript. Z.L., Y.Z. and H.X. wrote the manuscript. Y.Z. performed the structural characterizations, including those using GIXRD and TEM, with the assistance of J.Z. under the supervision of B.X. and N.-G.P. W.C. assessed performance in LED mode with photoluminescence, time-resolved photoluminescence and IPCE measurements under the supervision of Z.X. T.K. and P.R. analysed the results of the optoelectronic characterizations. B.L. conducted the in situ GIWAXS measurements with help from S.D. J.Z., X.G. and Q.W. assisted in the EXAFS and WAXS measurements. D.-H.K. assisted in the EXAFS measurements and analysis under the supervision of N.-G.P. H.Z. and Z.W. assisted in the synthesis of perovskite materials and PSP molecules. H.H. conducted the density functional theory computations. X.C. helped in the FIB-TEM measurements and analysis of the results. H.Z. conducted the SEM measurements. T.K., B.X., Z.X., S.D., N.-G.P., J.Y. and X.P. revised the manuscript. All authors discussed the results and commented on the manuscript.

Corresponding authors

Correspondence to Songyuan Dai, Nam-Gyu Park, Jiajiu Ye or Xu Pan.

Ethics declarations

Competing interests

A patent application related to the subject matter of this manuscript has been submitted (Application Nos. PCT/CN2022/143637 and 2022114089354) by the Hefei Institutes of Physical Science, Chinese Academy of Sciences.

Peer review

Peer review information

Nature thanks Zhaoning Song and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Illustration of peel-off method to expose the bottom of perovskite films.

We prepared sample comprising ITO/PTAA/Perovskite/Cu stacks for GIXRD measurements. PTAA (30 mg ml−1 in CB) was deposited as sacrifice layer to further expose bottom interface, and Cu layer was evaporated to ensure the completeness of perovskite films. After fabrication, the samples were immersed in a tank filled with CB for 20 min in a nitrogen glovebox. Along with dissolving of PTAA layer, an entire perovskite film together with Cu would float in CB. A clean quartz substrate was used for flipping and carrying the obtained films. The samples should be immersed in CB again for another 5 min to clear potential PTAA residues. Finally, the samples were dried naturally in nitrogen and for further characterizations of bottom interface. It should be noted that, immersion in CB within 1 h would not affect perovskite structure. To ensure the reliability for these results, the controlled trial was conducted with perovskite films deposited on quartz glass. We tested conventional XRD measurements for the perovskite surface, which demonstrated identical results of 1 h immersed perovskite films.

Extended Data Fig. 2 Analysis of synthesised (PbI2)[x](PSP)[y] complex.

a-b, The potential crystal structure of (PbI_2)x(PSP)y; c, The experimental XRD patterns of PSP, PbI_2 and (PbI_2)x(PSP)y and the complex simulations.

Extended Data Fig. 3 NMR results.

13C Chemical shift information of PSP and PSP(PbI_2) complex obtained from NMR measurements.

Extended Data Fig. 4 Band alignment analysis from ultraviolet photoelectron spectroscopy (UPS) depth profile.

a, Schematic diagram of method for depth profile UPS. b-c, UPS depth profiles of perovskite films (b) without and (c) with PSP introduction. The colour from the light to dark indicated spectra recorded after etching 0 nm, 400 nm and 700 nm, respectively.

Extended Data Fig. 5 Schematic diagram of band alignment extracted from UPS results.

The band structures of different depths have been combined through manually aligning the Fermi energy level, due to there were not actual contact junction of different detection region. a-b, Schematics of quasi-type I band alignment caused by FA-Cs phase segregation. In a solar cell with (a) p-i-n stacks, holes undergo extra energy loss at bottom region when it diffuses within the perovskite film, leading to charge carriers (holes) accumulate at this region. Ultimately break the carrier extraction equilibrium. The same situation would arise for the electrons in solar cells with (b) n-i-p configuration. c-d, Resultant flatten band alignment by PSP introduction in a device with (c) p-i-n configuration and (d) n-i-p configuration.

Extended Data Fig. 6 Transient absorption (TA) spectra.

a-b, TA spectra of perovskite films of (a) the reference and (b) PSP film deposited on quartz glass. c, Time-resolved fitting curve at 780 nm. We utilized TA measurements to examine carrier dynamics. In the PSP film, an enhancement in absorption variation (ΔA) was observed, corroborating the reduction in Shockley-Read-Hall (SRH) recombination due to the passivation of trap states. Time-resolved absorption at the ground-state bleach (GSB) of 780 nm revealed a decrease in the fast decay lifetime (τ_1) and an increase in the slow decay lifetime (τ_2). The five-fold reduction in τ_1 suggests effective passivation, while the nearly three-fold increase in τ_2 implies a seamless carrier diffusion within the perovskite (Supplementary Table 4). This finding aligns with the conclusion obtained from energy band alignment.

Extended Data Fig. 7 The plots of the first order derivative of EQE curve.

The bandgap was extracted from the first order derivative of EQE curve, which located at around 818 nm.

Extended Data Fig. 8 Additional stability assessments.

a, PCE evolution curves of PSCs under thermal cycling stress. The temperature cycling reliability were tested between the −40 °C~60 °C with duration time of 2 h. The initial PCE of the reference and the PSP device is 25.27% and 24.09%, respectively. b, Stability test followed ISOS L-2I protocol that performing MPP tracking under continuous 1 sun illumination at 65 °C in N_2 atmosphere. The solid lines represent for the average efficiency evolution among the eight individual device, and the shadow region represent for the efficiency evolution range during the tests. The average initial PCE of the reference and PSP device is 25.47% and 24.26%, respectively.

Supplementary information

Supplementary Information

Materials synthesis, Supplementary Figs. 1–30, Note 1 and Tables 1–7.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Liang, Z., Zhang, Y., Xu, H. et al. Homogenizing out-of-plane cation composition in perovskite solar cells. Nature 624, 557–563 (2023). https://doi.org/10.1038/s41586-023-06784-0

- Received: 01 January 2023

- Accepted: 25 October 2023

- Published: 01 November 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06784-0

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Stable and low-photovoltage-loss perovskite solar cells by multifunctional passivation

Article 05 July 2021

Guang Yang, Zhiwei Ren, … Gang Li

A universal close-space annealing strategy towards high-quality perovskite absorbers enabling efficient all-perovskite tandem solar cells

Article 21 July 2022

Changlei Wang, Yue Zhao, … Xiaofeng Li

Efficient lateral-structure perovskite single crystal solar cells with high operational stability

Article Open access 14 January 2020

Yilong Song, Weihui Bi, … Qingfeng Dong

Article Open access Published: 20 December 2023

Autonomous chemical research with large language models

Daniil A. Boiko,

Robert MacKnight,

Ben Kline &

…

Gabe Gomes

Show authors

Nature volume 624, pages 570–578 (2023)

- 45k Accesses

- 1 Citations

- 843 Altmetric

- Metrics details

Abstract

Transformer-based large language models are making significant strides in various fields, such as natural language processing1,2,3,4,5, biology6,7, chemistry8,9,10 and computer programming11,12. Here, we show the development and capabilities of Coscientist, an artificial intelligence system driven by GPT-4 that autonomously designs, plans and performs complex experiments by incorporating large language models empowered by tools such as internet and documentation search, code execution and experimental automation. Coscientist showcases its potential for accelerating research across six diverse tasks, including the successful reaction optimization of palladium-catalysed cross-couplings, while exhibiting advanced capabilities for (semi-)autonomous experimental design and execution. Our findings demonstrate the versatility, efficacy and explainability of artificial intelligence systems like Coscientist in advancing research.

Similar content being viewed by others

Main

Large language models (LLMs), particularly transformer-based models, are experiencing rapid advancements in recent years. These models have been successfully applied to various domains, including natural language1,2,3,4,5, biological6,7 and chemical research8,9,10 as well as code generation11,12. Extreme scaling of models[13], as demonstrated by OpenAI, has led to significant breakthroughs in the field1,14. Moreover, techniques such as reinforcement learning from human feedback[15] can considerably enhance the quality of generated text and the models’ capability to perform diverse tasks while reasoning about their decisions[16].

On 14 March 2023, OpenAI released their most capable LLM to date, GPT-4[14]. Although specific details about the model training, sizes and data used are limited in GPT-4’s technical report, OpenAI researchers have provided substantial evidence of the model’s exceptional problem-solving abilities. Those include—but are not limited to—high percentiles on the SAT and BAR examinations, LeetCode challenges and contextual explanations from images, including niche jokes[14]. Moreover, the technical report provides an example of how the model can be used to address chemistry-related problems.

Simultaneously, substantial progress has been made toward the automation of chemical research. Examples range from the autonomous discovery17,18 and optimization of organic reactions[19] to the development of automated flow systems20,21 and mobile platforms[22].

The combination of laboratory automation technologies with powerful LLMs opens the door to the development of a sought-after system that autonomously designs and executes scientific experiments. To accomplish this, we intended to address the following questions. What are the capabilities of LLMs in the scientific process? What degree of autonomy can we achieve? How can we understand the decisions made by autonomous agents?

In this work, we present a multi-LLMs-based intelligent agent (hereafter simply called Coscientist) capable of autonomous design, planning and performance of complex scientific experiments. Coscientist can use tools to browse the internet and relevant documentation, use robotic experimentation application programming interfaces (APIs) and leverage other LLMs for various tasks. This work has been done independently and in parallel to other works on autonomous agents23,24,25, with ChemCrow[26] serving as another example in the chemistry domain. In this paper, we demonstrate the versatility and performance of Coscientist in six tasks: (1) planning chemical syntheses of known compounds using publicly available data; (2) efficiently searching and navigating through extensive hardware documentation; (3) using documentation to execute high-level commands in a cloud laboratory; (4) precisely controlling liquid handling instruments with low-level instructions; (5) tackling complex scientific tasks that demand simultaneous use of multiple hardware modules and integration of diverse data sources; and (6) solving optimization problems requiring analyses of previously collected experimental data.

Coscientist system architecture

Coscientist acquires the necessary knowledge to solve a complex problem by interacting with multiple modules (web and documentation search, code execution) and by performing experiments. The main module (‘Planner’) has the goal of planning, based on the user input by invoking the commands defined below. The Planner is a GPT-4 chat completion instance serving the role of an assistant. The initial user input along with command outputs are treated as user messages to the Planner. System prompts (static inputs defining the LLMs’ goals) for the Planner are engineered1,27 in a modular fashion, described as four commands that define the action space: ‘GOOGLE’, ‘PYTHON’, ‘DOCUMENTATION’ and ‘EXPERIMENT’. The Planner calls on each of these commands as needed to collect knowledge. The GOOGLE command is responsible for searching the internet with the ‘Web searcher’ module, which is another LLM itself. The PYTHON command allows the Planner to perform calculations to prepare the experiment using a ‘Code execution’ module. The EXPERIMENT command actualizes ‘Automation’ through APIs described by the DOCUMENTATION module. Like GOOGLE, the DOCUMENTATION command provides information to the main module from a source, in this case documentation concerning the desired API. In this study, we have demonstrated the compatibility with the Opentrons Python API and the Emerald Cloud Lab (ECL) Symbolic Lab Language (SLL). Together, these modules make up Coscientist, which receives a simple plain text input prompt from the user (for example, “perform multiple Suzuki reactions”). This architecture is depicted in Fig. 1.

Fig. 1: The system’s architecture.

a, Coscientist is composed of multiple modules that exchange messages. Boxes with blue background represent LLM modules, the Planner module is shown in green, and the input prompt is in red. White boxes represent modules that do not use LLMs. b, Types of experiments performed to demonstrate the capabilities when using individual modules or their combinations. c, Image of the experimental setup with a liquid handler. UV-Vis, ultraviolet visible.

Furthermore, some of the commands can use subactions. The GOOGLE command is capable of transforming prompts into appropriate web search queries, running them against the Google Search API, browsing web pages and funneling answers back to the Planner. Similarly, the DOCUMENTATION command performs retrieval and summarization of necessary documentation (for example, robotic liquid handler or a cloud laboratory) for Planner to invoke the EXPERIMENT command.

The PYTHON command performs code execution (not reliant upon any language model) using an isolated Docker container to protect the users’ machine from any unexpected actions requested by the Planner. Importantly, the language model behind the Planner enables code to be fixed in case of software errors. The same applies to the EXPERIMENT command of the Automation module, which executes generated code on corresponding hardware or provides the synthetic procedure for manual experimentation.

Web search module

To demonstrate one of the functionalities of the Web Searcher module, we designed a test set composed of seven compounds to synthesize, as presented in Fig. 2a. The Web Searcher module versions are represented as ‘search-gpt-4’ and ‘search-gpt-3.5-turbo’. Our baselines include OpenAI’s GPT-3.5 and GPT-4, Anthropic’s Claude 1.3[28] and Falcon-40B-Instruct[29]—considered one of the best open-source models at the time of this experiment as per the OpenLLM leaderboard[30].

Fig. 2: Coscientist’s capabilities in chemical synthesis planning tasks.

a, Comparison of various LLMs on compound synthesis benchmarks. Error bars represent s.d. values. b, Two examples of generated syntheses of nitroaniline. c, Two example of generated syntheses of ibuprofen. UV, ultraviolet.

We prompted every model to provide a detailed compound synthesis, ranking the outputs on the following scale (Fig. 2):

- 5 for a very detailed and chemically accurate procedure description

- 4 for a detailed and chemically accurate description but without reagent quantities

- 3 for a correct chemistry description that does not include step-by-step procedure

- 2 for extremely vague or unfeasible descriptions

- 1 for incorrect responses or failure to follow instructions

All scores below 3 indicate task failure. It is important to note that all answers between 3 and 5 are chemically correct but offer varying levels of detail. Despite our attempts to better formalize the scale, labelling is inherently subjective and so, may be different between the labelers.

Across non-browsing models, the two versions of the GPT-4 model performed best, with Claude v.1.3 demonstrating similar performance. GPT-3 performed significantly worse, and Falcon 40B failed in most cases. All non-browsing models incorrectly synthesized ibuprofen (Fig. 2c). Nitroaniline is another example; although some generalization of chemical knowledge might inspire the model to propose direct nitration, this approach is not experimentally applicable as it would produce a mixture of compounds with a very minor amount of the product (Fig. 2b). Only the GPT-4 models occasionally provided the correct answer.

The GPT-4-powered Web Searcher significantly improves on synthesis planning. It reached maximum scores across all trials for acetaminophen, aspirin, nitroaniline and phenolphthalein (Fig. 2b). Although it was the only one to achieve the minimum acceptable score of three for ibuprofen, it performed lower than some of the other models for ethylacetate and benzoic acid, possibly because of the widespread nature of these compounds. These results show the importance of grounding LLMs to avoid ‘hallucinations’[31]. Overall, the performance of GPT-3.5-enabled Web Searcher trailed its GPT-4 competition, mainly because of its failure to follow specific instructions regarding output format.

Extending the Planner’s action space to leverage reaction databases, such as Reaxys[32] or SciFinder[33], should significantly enhance the system’s performance (especially for multistep syntheses). Alternatively, analysing the system’s previous statements is another approach to improving its accuracy. This can be done through advanced prompting strategies, such as ReAct[34], Chain of Thought[35] and Tree of Thoughts[36].

Documentation search module

Addressing the complexities of software components and their interactions is crucial for integrating LLMs with laboratory automation. A key challenge lies in enabling Coscientist to effectively utilize technical documentation. LLMs can refine their understanding of common APIs, such as the Opentrons Python API[37], by interpreting and learning from relevant technical documentation. Furthermore, we show how GPT-4 can learn how to programme in the ECL SLL.

Our approach involved equipping Coscientist with essential documentation tailored to specific tasks (as illustrated in Fig. 3a), allowing it to refine its accuracy in using the API and improve its performance in automating experiments.

Fig. 3: Overview of documentation search.

a, Prompt-to-code through ada embedding and distance-based vector search. b, Example of code for using OT-2’s heater–shaker module. c, Prompt-to-function/prompt-to-SLL (to symbolic laboratory language) through supplementation of documentation. d, Example of valid ECL SLL code for performing high-performance liquid chromatography (HPLC) experiments.

Information retrieval systems are usually based on two candidate selection approaches: inverted search index and vector database38,39,40,41. For the first one, each unique word in the search index is mapped to the documents containing it. At inference time, all documents containing words from a query are selected and ranked based on various manually defined formulas[42]. The second approach starts by embedding the documents with neural networks or as term frequency–inverse document frequency embedding vectors[43], followed by the construction of a vector database. Retrieval of similar vectors from this database occurs at inference time, usually using one of the approximate nearest neighbour search algorithms[44]. When strategies such as Transformer models are used, there are more chances to account for synonyms natively without doing synonym-based query expansion, as would be done in the first approach[45].

Following the second approach, all sections of the OT-2 API documentation were embedded using OpenAI’s ada model. To ensure proper use of the API, an ada embedding for the Planner’s query was generated, and documentation sections are selected through a distance-based vector search. This approach proved critical for providing Coscientist with information about the heater–shaker hardware module necessary for performing chemical reactions (Fig. 3b).

A greater challenge emerges when applying this approach to a more diverse robotic ecosystem, such as the ECL. Nonetheless, we can explore the effectiveness of providing information about the ECL SLL, which is currently unknown to the GPT-4 model. We conducted three separate investigations concerning the SLL: (1) prompt-to-function; (2) prompt-to-SLL; and (3) prompt-to-samples. Those investigations are detailed in Supplementary Information section ‘ECL experiments’.

For investigation 1, we provide the Docs searcher with a documentation guide from ECL pertaining to all available functions for running experiments[46]. Figure 3c summarizes an example of the user providing a simple prompt to the system, with the Planner receiving relevant ECL functions. In all cases, functions are correctly identified for the task.

Figure 3c,d continues to describe investigation 2, the prompt-to-SLL investigation. A single appropriate function is selected for the task, and the documentation is passed through a separate GPT-4 model to perform code retention and summarization. After the complete documentation has been processed, the Planner receives usage information to provide EXPERIMENT code in the SLL. For instance, we provide a simple example that requires the ‘ExperimentHPLC’ function. Proper use of this function requires familiarity with specific ‘Models’ and ‘Objects’ as they are defined in the SLL. Generated code was successfully executed at ECL; this is available in Supplementary Information. The sample was a caffeine standard sample. Other parameters (column, mobile phases, gradients) were determined by ECL’s internal software (a high-level description is in Supplementary Information section ‘HPLC experiment parameter estimation’). Results of the experiment are provided in Supplementary Information section ‘Results of the HPLC experiment in the cloud lab’. One can see that the air bubble was injected along with the analyte’s solution. This demonstrates the importance of development of automated techniques for quality control in cloud laboratories. Follow-up experiments leveraging web search to specify and/or refine additional experimental parameters (column chemistry, buffer system, gradient and so on) would be required to optimize the experimental results. Further details on this investigation are in Supplementary Information section ‘Analysis of ECL documentation search results’.

A separate prompt-to-samples investigation, investigation 3, was conducted by providing a catalogue of available samples, enabling the identification of relevant stock solutions that are on ECL’s shelves. To showcase this feature, we provide the Docs searcher module with all 1,110 Model samples from the catalogue. By simply providing a search term (for example, ‘Acetonitrile’), all relevant samples are returned. This is also available in Supplementary Information.

Controlling laboratory hardware

Access to documentation enables us to provide sufficient information for Coscientist to conduct experiments in the physical world. To initiate the investigation, we chose the Opentrons OT-2, an open-source liquid handler with a well-documented Python API. The ‘Getting Started’ page from its documentation was supplied to the Planner in the system prompt. Other pages were vectorized using the approach described above. For this investigation, we did not grant access to the internet (Fig. 4a).

Fig. 4: Robotic liquid handler control capabilities and integration with analytical tools.

a, Overview of Coscientist’s configuration. b, Drawing a red cross. c, Colouring every other row. d, Drawing a yellow rectangle. e, Drawing a blue diagonal.

We started with simple plate layout-specific experiments. Straightforward prompts in natural language, such as “colour every other line with one colour of your choice”, resulted in accurate protocols. When executed by the robot, these protocols closely resembled the requested prompt (Fig. 4b–e).

Ultimately, we aimed to assess the system’s ability to integrate multiple modules simultaneously. Specifically, we provided the ‘UVVIS’ command, which can be used to pass a microplate to plate reader working in the ultraviolet–visible wavelength range. To evaluate Coscientist’s capabilities to use multiple hardware tools, we designed a toy task; in 3 wells of a 96-well plate, three different colours are present—red, yellow and blue. The system must determine the colours and their positions on the plate without any prior information.

The Coscientist’s first action was to prepare small samples of the original solutions (Extended Data Fig. 1). Ultraviolet-visible measurements were then requested to be performed by the Coscientist (Supplementary Information section ‘Solving the colours problem’ and Supplementary Fig. 1). Once completed, Coscientist was provided with a file name containing a NumPy array with spectra for each well of the microplate. Coscientist subsequently generated Python code to identify the wavelengths with maximum absorbance and used these data to correctly solve the problem, although it required a guiding prompt asking it to think through how different colours absorb light.

Integrated chemical experiment design

We evaluated Coscientist’s ability to plan catalytic cross-coupling experiments by using data from the internet, performing the necessary calculations and ultimately, writing code for the liquid handler. To increase complexity, we asked Coscientist to use the OT-2 heater–shaker module released after the GPT-4 training data collection cutoff. The available commands and actions supplied to the Coscientist are shown in Fig. 5a. Although our setup is not yet fully automated (plates were moved manually), no human decision-making was involved.

Fig. 5: Cross-coupling Suzuki and Sonogashira reaction experiments designed and performed by Coscientist.

a, Overview of Coscientist’s configuration. b, Available compounds (DMF, dimethylformamide; DiPP, 2,6-diisopropylphenyl). c, Liquid handler setup. d, Solving the synthesis problem. e, Comparison of reagent selection performance with a large dataset. f, Comparison of reagent choices across multiple runs. g, Overview of justifications made when selecting various aryl halides. h, Frequency of visited URLs. i, Total ion current (TIC) chromatogram of the Suzuki reaction mixture (top panel) and the pure standard, mass spectra at 9.53 min (middle panel) representing the expected reaction product and mass spectra of the pure standard (bottom panel). j, TIC chromatogram of the Sonogashira reaction mixture (top panel) and the pure standard, mass spectra at 12.92 min (middle panel) representing the expected reaction product and mass spectra of the pure standard (bottom panel). Rel., relative.

The test challenge for Coscientist’s complex chemical experimentation capabilities was designed as follows. (1) Coscientist is provided with a liquid handler equipped with two microplates (source and target plates). (2) The source plate contains stock solutions of multiple reagents, including phenyl acetylene and phenylboronic acid, multiple aryl halide coupling partners, two catalysts, two bases and the solvent to dissolve the sample (Fig. 5b). (3) The target plate is installed on the OT-2 heater–shaker module (Fig. 5c). (4) Coscientist’s goal is to successfully design and perform a protocol for Suzuki–Miyaura and Sonogashira coupling reactions given the available resources.

To start, Coscientist searches the internet for information on the requested reactions, their stoichiometries and conditions (Fig. 5d). The correct coupling partners are selected for the corresponding reactions. Designing and performing the requested experiments, the strategy of Coscientist changes among runs (Fig. 5f). Importantly, the system does not make chemistry mistakes (for instance, it never selects phenylboronic acid for the Sonogashira reaction). Interestingly, the base DBU (1,8-diazabicyclo[5.4.0]undec-7-ene) is selected more often with the PEPPSI–IPr (PEPPSI, pyridine-enhanced precatalyst preparation stabilization and initiation; IPr, 1,3-bis(2,6-diisopropylphenyl)imidazol-2-ylidene) complex, with that preference switching in Sonogashira reaction experiments; likewise, bromobenzene is chosen more often for Suzuki than for Sonogashira couplings. Additionally, the model can provide justifications on specific choices (Fig. 5g), demonstrating the ability to operate with concepts such as reactivity and selectivity (more details are in Supplementary Information section ‘Analysis of behaviour across multiple runs’). This capability highlights a potential future use case to analyse the reasoning of the LLMs used by performing experiments multiple times. Although the Web Searcher visited various websites (Fig. 5h), overall Coscientist retrieves Wikipedia pages in approximately half of cases; notably, American Chemical Society and Royal Society of Chemistry journals are amongst the top five sources.

Coscientist then calculates the required volumes of all reactants and writes a Python protocol for running the experiment on the OT-2 robot. However, an incorrect heater–shaker module method name was used. Upon making this mistake, Coscientist uses the Docs searcher module to consult the OT-2 documentation. Next, Coscientist modifies the protocol to a corrected version, which ran successfully (Extended Data Fig. 2). Subsequent gas chromatography–mass spectrometry analysis of the reaction mixtures revealed the formation of the target products for both reactions. For the Suzuki reaction, there is a signal in the chromatogram at 9.53 min where the mass spectra match the mass spectra for biphenyl (corresponding molecular ion mass-to-charge ratio and fragment at 76 Da) (Fig. 5i). For the Sonogashira reaction, we see a signal at 12.92 min with a matching molecular ion mass-to-charge ratio; the fragmentation pattern also looks very close to the one from the spectra of the reference compound (Fig. 5j). Details are in Supplementary Information section ‘Results of the experimental study’.

Although this example requires Coscientist to reason on which reagents are most suitable, our experimental capabilities at that point limited the possible compound space to be explored. To address this, we performed several computational experiments to evaluate how a similar approach can be used to retrieve compounds from large compound libraries[47]. Figure 5e shows Coscientist’s performance across five common organic transformations, with outcomes depending on the queried reaction and its specific run (the GitHub repository has more details). For each reaction, Coscientist was tasked with generating reactions for compounds from a simplified molecular-input line-entry system (SMILES) database. To achieve the task, Coscientist uses web search and code execution with the RDKit chemoinformatics package.

Chemical reasoning capabilities

The system demonstrates appreciable reasoning capabilities, enabling the request of necessary information, solving of multistep problems and generation of code for experimental design. Some researchers believe that the community is only starting to understand all the capabilities of GPT-4 (ref. [48]). OpenAI has shown that GPT-4 could rely on some of those capabilities to take actions in the physical world during their initial red team testing performed by the Alignment Research Center[14].

One of the possible strategies to evaluate an intelligent agent’s reasoning capabilities is to test if it can use previously collected data to guide future actions. Here, we focused on the multi-variable design and optimization of Pd-catalysed transformations, showcasing Coscientist’s abilities to tackle real-world experimental campaigns involving thousands of examples. Instead of connecting LLMs to an optimization algorithm as previously done by Ramos et al.[49], we aimed to use Coscientist directly.

We selected two datasets containing fully mapped reaction condition spaces where yield was available for all combinations of variables. One is a Suzuki reaction dataset collected by Perera et al.[50], where these reactions were performed in flow with varying ligands, reagents/bases and solvents (Fig. 6a). Another is Doyle’s Buchwald–Hartwig reaction dataset[51] (Fig. 6e), where variations in ligands, additives and bases were recorded. At this point, any reaction proposed by Coscientist would be within these datasets and accessible as a lookup table.

Fig. 6: Results of the optimization experiments.

a, A general reaction scheme from the flow synthesis dataset analysed in c and d. b, The mathematical expression used to calculate normalized advantage values. c, Comparison of the three approaches (GPT-4 with prior information, GPT-4 without prior information and GPT-3.5 without prior information) used to perform the optimization process. d, Derivatives of the NMA and normalized advantage values evaluated in c, left and centre panels. e, Reaction from the C–N cross-coupling dataset analysed in f and g. f, Comparison of two approaches using compound names and SMILES string as compound representations. g, Coscientist can reason about electronic properties of the compounds, even when those are represented as SMILES strings. DMSO, dimethyl sulfoxide.

We designed the Coscientist’s chemical reasoning capabilities test as a game with the goal of maximizing the reaction yield. The game’s actions consisted of selecting specific reaction conditions with a sensible chemical explanation while listing the player’s observations about the outcome of the previous iteration. The only hard rule was for the player to provide its actions written in JavaScript Object Notation (JSON) format. If the JSON file could not be parsed, the player is alerted of its failure to follow the specified data format. The player had a maximum of 20 iterations (accounting for 5.2% and 6.9% of the total space for the first and second datasets, respectively) to finish the game.

We evaluate Coscientist’s performance using the normalized advantage metric (Fig. 6b). Advantage is defined as the difference between a given iteration yield and the average yield (advantage over a random strategy). Normalized advantage measures the ratio between advantage and maximum advantage (that is, the difference between the maximum and average yield). The normalized advantage metric has a value of one if the maximum yield is reached, zero if the system exhibits completely random behaviour and less than zero if the performance at this step is worse than random. An increase in normalized advantage over each iteration demonstrates Coscientist’s chemical reasoning capabilities. The best result for a given iteration can be evaluated using the normalized maximum advantage (NMA), which is the normalized value of the maximum advantage achieved until the current step. As NMA cannot decrease, the valuable observations come in the form of the rate of its increase and its final point. Finally, during the first step, the values for NMA and normalized advantage equal each other, portraying the model’s prior knowledge (or lack thereof) without any data being collected.

For the Suzuki dataset, we compared three separate approaches: (1) GPT-4 with prior information included in the prompt (which consisted of 10 yields from random combinations of reagents); (2) GPT-4; or (3) GPT-3.5 without any prior information (Fig. 6c). When comparing GPT-4 with the inclusion and exclusion of prior information, it is clear that the initial guess for the former scenario is better, which aligns with our expectations considering the provided information about the system’s reactivity. Notably, when excluding prior information, there are some poor initial guesses, whereas there are none when the model has prior information. However, at the limit, the models converge to the same NMA. The GPT-3.5 model plots have a very limited number of data points, primarily because of its inability to output messages in the correct JSON schema as requested in the prompt. It is unclear if the GPT-4 training data contain any information from these datasets. If so, one would expect that the initial model guess would be better than what we observed.

The normalized advantage values increase over time, suggesting that the model can effectively reuse the information obtained to provide more specific guidance on reactivity. Evaluating the derivative plots (Fig. 6d) does not show any significant difference between instances with and without the input of prior information.

There are many established optimization algorithms for chemical reactions. In comparison with standard Bayesian optimization[52], both GPT-4-based approaches show higher NMA and normalized advantage values (Fig. 6c). A detailed overview of the exact Bayesian optimization strategy used is provided in Supplementary Information section ‘Bayesian optimization procedure’. It is observed that Bayesian optimization’s normalized advantage line stays around zero and does not increase over time. This may be caused by different exploration/exploitation balance for these two approaches and may not be indicative of their performance. For this purpose, the NMA plot should be used. Changing the number of initial samples does not improve the Bayesian optimization trajectory (Extended Data Fig. 3a). Finally, this performance trend is observed for each unique substrate pairings (Extended Data Fig. 3b).

For the Buchwald–Hartwig dataset (Fig. 6e), we compared a version of GPT-4 without prior information operating over compound names or over compound SMILES strings. It is evident that both instances have very similar performance levels (Fig. 6f). However, in certain scenarios, the model demonstrates the ability to reason about the reactivity of these compounds simply by being provided their SMILES strings (Fig. 6g).

Discussion

In this paper, we presented a proof of concept for an artificial intelligent agent system capable of (semi-)autonomously designing, planning and multistep executing scientific experiments. Our system demonstrates advanced reasoning and experimental design capabilities, addressing complex scientific problems and generating high-quality code. These capabilities emerge when LLMs gain access to relevant research tools, such as internet and documentation search, coding environments and robotic experimentation platforms. The development of more integrated scientific tools for LLMs has potential to greatly accelerate new discoveries.

The development of new intelligent agent systems and automated methods for conducting scientific experiments raises potential concerns about the safety and potential dual-use consequences, particularly in relation to the proliferation of illicit activities and security threats. By ensuring the ethical and responsible use of these powerful tools, we are continuing to explore the vast potential of LLMs in advancing scientific research while mitigating the risks associated with their misuse. A brief dual-use study of Coscientist is provided in Supplementary Information section ‘Safety implications: Dual-use study’.

Technology use disclosure

The writing of the preprint version of this manuscript was assisted by ChatGPT (specifically, GPT-4 being used for grammar and typos). All authors have read, corrected and verified all information presented in this manuscript and Supplementary Information.

Data availability

Examples of the experiments discussed in the text are provided in the Supplementary Information. Because of safety concerns, data, code and prompts will be only fully released after the development of US regulations in the field of artificial intelligence and its scientific applications. Nevertheless, the outcomes of this work can be reproduced using actively developed frameworks for autonomous agent development. The reviewers had access to the web application and were able to verify any statements related to this work. Moreover, we provide a simpler implementation of the described approach, which, although it may not produce the same results, allows for deeper understanding of the strategies used in this work.

Code availability

Simpler implementation as well as generated outputs used for quantitative analysis are provided at https://github.com/gomesgroup/coscientist.

References

1. Brown, T. et al. in Advances in Neural Information Processing Systems Vol. 33 (eds Larochelle, H. et al.) 1877–1901 (Curran Associates, 2020).

2. Thoppilan, R. et al. LaMDA: language models for dialog applications. Preprint at https://arxiv.org/abs/2201.08239 (2022).

3. Touvron, H. et al. LLaMA: open and efficient foundation language models. Preprint at https://arxiv.org/abs/2302.13971 (2023).

4. Hoffmann, J. et al. Training compute-optimal large language models. In Advances in Neural Information Processing Systems 30016–30030 (NeurIPS, 2022).

5. Chowdhery, A. et al. PaLM: scaling language modeling with pathways. J. Mach. Learn. Res. 24, 1–113 (2022).

6. Lin, Z. et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379, 1123–1130 (2023).

7. Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Brief Bioinform. 23, bbac409 (2022).

8. Irwin, R., Dimitriadis, S., He, J. & Bjerrum, E. J. Chemformer: a pre-trained transformer for computational chemistry. Mach. Learn. Sci. Technol. 3, 015022 (2022).

9. Kim, H., Na, J. & Lee, W. B. Generative chemical transformer: neural machine learning of molecular geometric structures from chemical language via attention. J. Chem. Inf. Model. 61, 5804–5814 (2021).

10. Jablonka, K. M., Schwaller, P., Ortega-Guerrero, A. & Smit, B. Leveraging large language models for predictive chemistry. Preprint at https://chemrxiv.org/engage/chemrxiv/article-details/652e50b98bab5d2055852dde (2023).

11. Xu, F. F., Alon, U., Neubig, G. & Hellendoorn, V. J. A systematic evaluation of large language models of code. In Proc. 6th ACM SIGPLAN International Symposium on Machine Programming 1–10 (ACM, 2022).

12. Nijkamp, E. et al. CodeGen: an open large language model for code with multi-turn program synthesis. In Proc. 11th International Conference on Learning Representations (ICLR, 2022).

13. Kaplan, J. et al. Scaling laws for neural language models. Preprint at https://arxiv.org/abs/2001.08361 (2020).

14. OpenAI. GPT-4 Technical Report (OpenAI, 2023).

15. Ziegler, D. M. et al. Fine-tuning language models from human preferences. Preprint at https://arxiv.org/abs/1909.08593 (2019).

16. Ouyang, L. et al. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 27730–27744 (NeurIPS, 2022).

17. Granda, J. M., Donina, L., Dragone, V., Long, D.-L. & Cronin, L. Controlling an organic synthesis robot with machine learning to search for new reactivity. Nature 559, 377–381 (2018).

18. Caramelli, D. et al. Discovering new chemistry with an autonomous robotic platform driven by a reactivity-seeking neural network. ACS Cent. Sci. 7, 1821–1830 (2021).

19. Angello, N. H. et al. Closed-loop optimization of general reaction conditions for heteroaryl Suzuki–Miyaura coupling. Science 378, 399–405 (2022).

20. Adamo, A. et al. On-demand continuous-flow production of pharmaceuticals in a compact, reconfigurable system. Science 352, 61–67 (2016).

21. Coley, C. W. et al. A robotic platform for flow synthesis of organic compounds informed by AI planning. Science 365, eaax1566 (2019).

22. Burger, B. et al. A mobile robotic chemist. Nature 583, 237–241 (2020).

23. Auto-GPT: the heart of the open-source agent ecosystem. GitHub https://github.com/Significant-Gravitas/AutoGPT (2023).

24. BabyAGI. GitHub https://github.com/yoheinakajima/babyagi (2023).

25. Chase, H. LangChain. GitHub https://github.com/langchain-ai/langchain (2023).

26. Bran, A. M., Cox, S., White, A. D. & Schwaller, P. ChemCrow: augmenting large-language models with chemistry tools. Preprint at https://arxiv.org/abs/2304.05376 (2023).

27. Liu, P. et al. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. ACM Comput. Surv. 55, 195 (2021).

28. Bai, Y. et al. Constitutional AI: harmlessness from AI feedback. Preprint at https://arxiv.org/abs/2212.08073 (2022).

29. Falcon LLM. TII https://falconllm.tii.ae (2023).

30. Open LLM Leaderboard. Hugging Face https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard (2023).

31. Ji, Z. et al. Survey of hallucination in natural language generation. ACM Comput. Surv. 55, 248 (2023).

32. Reaxys https://www.reaxys.com (2023).

33. SciFinder https://scifinder.cas.org (2023).

34. Yao, S. et al. ReAct: synergizing reasoning and acting in language models. In Proc.11th International Conference on Learning Representations (ICLR, 2022).

35. Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 24824–24837 (NeurIPS, 2022).

36. Long, J. Large language model guided tree-of-thought. Preprint at https://arxiv.org/abs/2305.08291 (2023).

37. Opentrons Python Protocol API. Opentrons https://docs.opentrons.com/v2/ (2023).

38. Tu, Z. et al. Approximate nearest neighbor search and lightweight dense vector reranking in multi-stage retrieval architectures. In Proc. 2020 ACM SIGIR on International Conference on Theory of Information Retrieval 97–100 (ACM, 2020).

39. Lin, J. et al. Pyserini: a python toolkit for reproducible information retrieval research with sparse and dense representations. In Proc. 44th International ACM SIGIR Conference on Research and Development in Information Retrieval 2356–2362 (ACM, 2021).

40. Qadrud-Din, J. et al. Transformer based language models for similar text retrieval and ranking. Preprint at https://arxiv.org/abs/2005.04588 (2020).

41. Paper QA. GitHub https://github.com/whitead/paper-qa (2023).

42. Robertson, S. & Zaragoza, H. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retrieval 3, 333–389 (2009).

43. Data Mining. Mining of Massive Datasets (Cambridge Univ., 2011).

44. Johnson, J., Douze, M. & Jegou, H. Billion-scale similarity search with GPUs. IEEE Trans. Big Data 7, 535–547 (2021).

45. Vechtomova, O. & Wang, Y. A study of the effect of term proximity on query expansion. J. Inf. Sci. 32, 324–333 (2006).

46. Running experiments. Emerald Cloud Lab https://www.emeraldcloudlab.com/guides/runningexperiments (2023).

47. Sanchez-Garcia, R. et al. CoPriNet: graph neural networks provide accurate and rapid compound price prediction for molecule prioritisation. Digital Discov. 2, 103–111 (2023).

48. Bubeck, S. et al. Sparks of artificial general intelligence: early experiments with GPT-4. Preprint at https://arxiv.org/abs/2303.12712 (2023).

49. Ramos, M. C., Michtavy, S. S., Porosoff, M. D. & White, A. D. Bayesian optimization of catalysts with in-context learning. Preprint at https://arxiv.org/abs/2304.05341 (2023).

50. Perera, D. et al. A platform for automated nanomole-scale reaction screening and micromole-scale synthesis in flow. Science 359, 429–434 (2018).

51. Ahneman, D. T., Estrada, J. G., Lin, S., Dreher, S. D. & Doyle, A. G. Predicting reaction performance in C–N cross-coupling using machine learning. Science 360, 186–190 (2018).

52. Hickman, R. et al. Atlas: a brain for self-driving laboratories. Preprint at https://chemrxiv.org/engage/chemrxiv/article-details/64f6560579853bbd781bcef6 (2023).

Acknowledgements

We thank the following Carnegie Mellon University Chemistry groups for their assistance with providing the chemicals needed for the Coscientist’s experiments: Sydlik, Garcia Borsch, Matyjaszewski and Ly. We give special thanks to the Noonan group (K. Noonan and D. Sharma) for providing access to chemicals and gas chromatography–mass spectrometry analysis. We also thank the team at Emerald Cloud Lab (with special attention to Y. Benslimane, H. Gronlund, B. Smith and B. Frezza) for assisting us with parsing their documentation and executing experiments. G.G. is grateful to the Carnegie Mellon University Cloud Lab Initiative led by the Mellon College of Science for its vision of the future of physical sciences. G.G. thanks Carnegie Mellon University; the Mellon College of Sciences and its Department of Chemistry; and the College of Engineering and its Department of Chemical Engineering for the start-up support. D.A.B. was partially funded by the National Science Foundation Center for Chemoenzymatic Synthesis (Grant no. 2221346). R.M. was funded by the National Science Foundation Center for Computer-Assisted Synthesis (Grant no. 2202693).

Author information

Authors and Affiliations

1. Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA

Daniil A. Boiko, Robert MacKnight & Gabe Gomes

2. Emerald Cloud Lab, South San Francisco, CA, USA

Ben Kline

3. Department of Chemistry, Carnegie Mellon University, Pittsburgh, PA, USA

Gabe Gomes

4. Wilton E. Scott Institute for Energy Innovation, Carnegie Mellon University, Pittsburgh, PA, USA

Gabe Gomes

Contributions

D.A.B. designed the computational pipeline and developed the ‘Planner’, ‘Web searcher’ and ‘Code execution’ modules. R.M. assisted in designing the computational pipeline and developed the ‘Docs searcher’ module. B.K. analysed the behaviours of the Docs searcher module to enable Coscientist to produce experiment code in Emerald Cloud Lab’s Symbolic Lab Language. D.A.B. assisted and oversaw Coscientist’s chemistry experiments. D.A.B., R.M. and G.G. designed and performed initial computational safety studies. D.A.B. designed and graded Coscientist’s synthesis capabilities study. D.A.B. co-designed with G.G. and performed the optimization experiments. R.M. performed the large compound library experiment and Bayesian optimization baseline runs. G.G. designed the concepts, performed preliminary studies and supervised the project. D.A.B., R.M. and G.G. wrote this manuscript.

Corresponding author

Correspondence to Gabe Gomes.

Ethics declarations

Competing interests

G.G. is part of the AI Scientific Advisory Board of Emerald Cloud Lab. Experiments and conclusions in this manuscript were made before G.G.’s appointment to this role. B.K. is an employee of Emerald Cloud Lab. D.A.B. and G.G. are co-founders of aithera.ai, a company focusing on responsible use of artificial intelligence for research.

Peer review

Peer review information

Nature thanks Sebastian Farquhar, Tiago Rodrigues and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Using UV-Vis and liquid handler to solve food colouring identification problem.

Guiding prompt in the third message is shown in bold. In the first message the user prompt is provided, then code for sample preparation is generated, resulting data is provided as NumPy array, which is then analysed to give the final answer.

Extended Data Fig. 2 Code, generated by Coscientist.

The generated code can be split into the following steps: defining metadata for the method, loading labware modules, setting up the liquid handler, performing required reagent transfers, setting up the heater-shaker module, running the reaction, and turning the module off.

Extended Data Fig. 3 Additional results on comparison with Bayesian optimization.

a, GPT-4 models compared with Bayesian optimization performed starting with different number of initial samples. b, Compound-by-compound comparison of differences between advantages.

Supplementary information

Supplementary Information

Supplementary Text and Figs. 1–3.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Boiko, D.A., MacKnight, R., Kline, B. et al. Autonomous chemical research with large language models. Nature 624, 570–578 (2023). https://doi.org/10.1038/s41586-023-06792-0

- Received: 20 April 2023

- Accepted: 27 October 2023

- Published: 20 December 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06792-0

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

- Large language models direct automated chemistry laboratory

- Ana Laura Dias

- Tiago Rodrigues

Nature (2023)

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Mathematical discoveries from program search with large language models

Article 14 December 2023

Bernardino Romera-Paredes, Mohammadamin Barekatain, … Alhussein Fawzi

De novo design of high-affinity binders of bioactive helical peptides

Article 18 December 2023

Susana Vázquez Torres, Philip J. Y. Leung, … David Baker

Assembly theory explains and quantifies selection and evolution

Article Open access 04 October 2023

Abhishek Sharma, Dániel Czégel, … Leroy Cronin

Large language models direct automated chemistry laboratory

- Ana Laura Dias

- Tiago Rodrigues

Nature News & Views 20 Dec 2023

Article Open access Published: 06 December 2023

Biological carbon pump estimate based on multidecadal hydrographic data

Wei-Lei Wang,

Weiwei Fu,

Frédéric A. C. Le Moigne,

Robert T. Letscher,

Yi Liu,

Jin-Ming Tang &

…

François W. Primeau

Show authors

Nature volume 624, pages 579–585 (2023)

- 12k Accesses

- 219 Altmetric

- Metrics details

Abstract

The transfer of photosynthetically produced organic carbon from surface to mesopelagic waters draws carbon dioxide from the atmosphere[1]. However, current observation-based estimates disagree on the strength of this biological carbon pump (BCP)[2]. Earth system models (ESMs) also exhibit a large spread of BCP estimates, indicating limited representations of the known carbon export pathways[3]. Here we use several decades of hydrographic observations to produce a top-down estimate of the strength of the BCP with an inverse biogeochemical model that implicitly accounts for all known export pathways. Our estimate of total organic carbon (TOC) export at 73.4 m (model euphotic zone depth) is 15.00 ± 1.12 Pg C year−1, with only two-thirds reaching 100 m depth owing to rapid remineralization of organic matter in the upper water column. Partitioned by sequestration time below the euphotic zone, τ, the globally integrated organic carbon production rate with τ > 3 months is 11.09 ± 1.02 Pg C year−1, dropping to 8.25 ± 0.30 Pg C year−1 for τ > 1 year, with 81% contributed by the non-advective-diffusive vertical flux owing to sinking particles and vertically migrating zooplankton. Nevertheless, export of organic carbon by mixing and other fluid transport of dissolved matter and suspended particles remains regionally important for meeting the respiratory carbon demand. Furthermore, the temperature dependence of the sequestration efficiency inferred from our inversion suggests that future global warming may intensify the recycling of organic matter in the upper ocean, potentially weakening the BCP.

Similar content being viewed by others

Main

The downward flux of biologically produced organic carbon draws CO_2 out of the atmosphere, contributing to the maintenance of a vertical gradient of dissolved inorganic carbon (DIC) in the ocean[4]. Much of the primary production occurring in sunlit waters is respired in surface waters without greatly affecting the partitioning of CO_2 between the atmosphere and ocean[2]. Attention, therefore, focuses on the fraction of the net primary production (NPP) exported to deeper waters before being respired. Considerable effort has focused on discovering the processes responsible for the regional differences in the so-called ef-ratio2,5,6,7, defined as export, or new production, divided by NPP (e-ratio and f-ratio, respectively). Oceanographers rely on empirical relationships between the ef-ratio and satellite-based measurements of NPP and sea surface temperature (SST) to obtain global-scale export patterns. Unfortunately, different versions of these empirical relationships, which typically assume that the ef-ratio is positively correlated with NPP and negatively correlated with SST, produce globally integrated estimates of carbon export that can vary by as much as a factor of three (5–12 Pg C year−1), although part of the spread may be caused by the different choices of export depth and data-coverage issues2,8. Moreover, several field observations in highly productive regions such as the Southern Ocean often contradict the assumption that the ef-ratio is positively correlated with NPP (refs. 9,10,11,12,13).

One cause for these discrepancies is that most observations provide only snapshots of the ocean at the time of collection, whereas episodic signals may be missed in models. Another explanation is that empirical algorithms focus almost entirely on the contribution from sinking particles, neglecting possibly important contributions from vertically migrating zooplankton and the transport of dissolved and non-sinking particulate organic carbon (POC) by subducting and overturning water masses (also known as the particle injection pump[1]). For instance, support for the importance of non-sinking particles is provided by the work of Emerson[14], who estimated annual net community production (ANCP) at three time-series sites (ALOHA, Hawaii Ocean Time-series station; BATS, Bermuda Atlantic Time-series Study; and OSP, Ocean Station Papa). He found that sinking POC flux is 3–4 times lower than required by mass-balance analyses. Indeed, Boyd et al.[1] suggest that non-gravitational export pathways acting on suspended particles can account for as much carbon export as the gravitational carbon pump, although the strength of these export pathways remains uncertain.

Regardless of how organic matter produced in surface waters is transferred to depth (gravitationally or not, in particulate form or not), most of it eventually remineralizes to inorganic carbon and nutrients, consuming dissolved oxygen (O_2) along the way. The resulting imprint on the dissolved oxygen, inorganic carbon, dissolved organic carbon (DOC) and dissolved organic phosphate (DIP) concentrations, for which there exist global databases (GLODAPv2.2021 (ref. [15]) and an updated DOC database[16]) collected over several decades of ship-based campaigns (Extended Data Figs. 1 and 2), allows us to infer the total carbon export and its regional variations. Here, using an inverse biogeochemical model for the cycling of phosphorus (P), carbon (C) and oxygen (O) (Fig. 1), we estimate the global distribution of the export flux separated into contributions from advective-diffusive flux, which encompasses fluxes mediated by physical transports such as the mixed-layer pump[17] and the subduction pump[18], and DOC contribution to the biological pump19,20, and non-advective-diffusive vertical flux that includes contributions from the gravitational pump[1], zooplankton migration pump[21] and seasonal lipid pump[22]. The model has 21 adjustable parameters whose values are constrained from global databases of DIP, total alkalinity (ALK), DIC, DOC and O_2 using a Bayesian inversion procedure.

Fig. 1: Schematic representations of phosphorus, carbon and oxygen cycling in the ocean.

a, Phosphorus cycling. b, Carbon cycling. c, Oxygen cycling. The dissolved tracers, oxygen (O_2), DIP and DIC, DOP and DOC, and ALK are transported by advection and diffusion. POP and POC are transported vertically downward and remineralized in the water column. The downward particulate flux attenuation is modelled so as to produce a power-law depth dependence. PIC also sinks and dissolves in the water column. Its downward flux attenuation is modelled so as to produce an exponential depth dependence. DIC and O_2 experience sea-to-air gas fluxes in the surface ocean, which are represented by the coiled arrows above DIC and O_2, respectively (FCO2 and FO2). The DIC and ALK concentrations are influenced by evaporation and precipitation. Thus, a virtual flux (coiled arrows above DIC and below ALK; FvDIC and FvALK, respectively) is applied to DIC and ALK to account for the concentrating and diluting effects of precipitation and evaporation. The solid lines in the schematic for the carbon-cycle model represent the pools that are connected by means of sink–source relationships. The dashed lines indicate the pools that are related by indirect source–sink relationships. For example, the formation of POC does not directly release ALK but instead changes the chemical form of nitrogen, which leads to changes in the ALK. See Methods for the symbol definitions and the Supplementary Information for their numerical values. κ[TOC] in the O_2 model represents the remineralization of TOC (see Methods for its full expression).

After fitting the 21 parameters (Extended Data Table 1), the model captures most of the spatial variance in the three-dimensional distribution of DIP (R^2 = 0.93, n = 76,480), DIC (R^2 = 0.94, n = 63,085), ALK (R^2 = 0.87, n = 59,093), O_2 (R^2 = 0.88, n = 83,732) and total DOC (R^2 = 0.80, n = 21,295) (Extended Data Figs. 3 and 4). What distinguishes our model from previous inverse models23,24 is the small number of adjustable parameters and the simultaneous use of several tracers to constrain the inversion. In particular, our inverse model uses DIC measurements, which provide the most natural constraint on the BCP. Previous inverse models did not use DIC observations to avoid the need to explicitly model the transient anthropogenic carbon signal in the hydrographic DIC dataset. Here we explicitly simulated the transient DIC signal and found that it contributes an approximately 20% decline in the vertical DIC gradient produced by the biological pump (Extended Data Fig. 5c; Methods). Furthermore, by combining ALK and DIC data with an accurate representation of the anthropogenic DIC signal, our model captures the respiration of organic carbon not oxidized by O_2 (ref. [25]). In a sensitivity test in which we followed refs. 23,24 by using only O_2 and DOC to constrain the model, we found a substantial deterioration of the fits to other tracers (Extended Data Fig. 3). Our model results are further validated using deep-water POC fluxes measured using sediment traps at time-series stations and the ANCP estimated using several geochemical tracers (see text below). The model is able to match different satellite-based NPP products (CbPM and CAFE) by adjusting labile DOC production (Extended Data Fig. 6; Methods) without greatly affecting the goodness of fit to tracers or the estimated carbon fluxes with residence times greater than about 1 year.

Organic carbon fluxes

In our model, which has a horizontal mesh resolution of 2° × 2° and 24 vertical layers, we define export according to the timescale for the vertical transfer of the organic carbon. Fluxes by fast-sinking POC (gravitational pump) and vertical zooplankton movements (vertical migration pump and seasonal lipid pump), which transport carbon vertically with no appreciable lateral transport, are assigned to non-advective-diffusive vertical export. Fluxes induced by organic carbon detrainment caused by changes of mixed-layer depth (mixed-layer-depth pump)[17] and physical subduction (subduction pump)[18] are assigned to advective-diffusive export. We note that, although the DOC pool of our model includes what would be characterized as suspended POC in field measurements and therefore is missing from the DOC measurement database, we believe that the difference is negligible for most of the ocean because the concentration of suspended POC is much lower (less than a few μM) than that of DOC (dozens of μM)[19]. We infer the strength and distribution of the total BCP from tracer distributions, which avoids counting the same export pathways several times[1].

Globally integrated, our estimated non-advective-diffusive vertical flux, which is calculated by integrating POC remineralization below 73.4 m, the euphotic zone depth of the model, is 10.63 ± 0.14 Pg C year−1. For comparison, simulated export production in the Coupled Model Intercomparison Project Phase 5 (CMIP5) models ranged from approximately 4.5 to 7.5 Pg C year−1 (ref. [26]). The spread in the newer CMIP6 models is even larger, ranging from approximately 5 to 12 Pg C year−1 at about 100 m (ref. [3]). Our most probable estimate is almost triple that obtained from the 234Th method (4 Pg C year−1)[2]. That our estimated export flux is larger than the 234Th-based estimate is not surprising because our flux includes not only the gravitational pump but also fluxes mediated by zooplankton migration. By contrast, the 234Th method constrains only the flux of sinking POC.

Geographically, our estimated non-advective-diffusive vertical export rate is high in coastal upwelling regions, the Southern Ocean convergent zones, subpolar North Pacific and Atlantic oceans and low in the subtropics (Fig. 2 and Extended Data Fig. 7). The non-advective-diffusive vertical flux is consistent with measurements from deep-water sediment traps[27] at ocean stations ALOHA, OSP, BATS and CARIACO, in which extensive measurements exist (Fig. 2), even though such POC-flux measurements only partially include contributions from zooplankton migration (faecal pellets). The similarity may be because the migration pump is weak in oligotrophic oceans24,28,29, in which ALOHA and BATS are located. For the mid-latitude OSP site, our estimates for the upper 200 m are higher than the median values of in situ sediment-trap measurements, probably because of the contribution from the migration pump. For the coastal CARIACO station, the higher fluxes from sediment-trap measurements have several possible explanations. First, our model may not have adequate resolution. Second, the bias may be the result of blooms, which may be poorly represented in our climatological-mean model. Last, sediment traps may overestimate particle flux in coastal regions because of augmented ‘statistical funnels’ of particle collection[30] or catchment of large aggregates mediated by a range of physical and biological processes[1].

Fig. 2: Non-advective-diffusive vertical flux.

a, Contour plot of non-advective-diffusive vertical flux (mg C m−2 day−1) exiting the base of the euphotic zone, with the location of the four ocean stations (OSP, ALOHA, BATS and CARIACO) marked with black stars. b–e, Model-derived non-advective-diffusive vertical flux at different depths compared with trap-determined POC flux. The box plots represent sediment trap and the green circles are model predictions, with error bars representing ±1σ derived from different model configurations. The box plots summarize the distributions of in situ measurements of POC flux, which show the 25th, 50th and 75th percentiles binned according to the POC flux. The whiskers cover 99.3% of the data, with the remaining points shown as red crosses. In b–e, the sediment-trap data presented are multiyear collections covering a sampling period of 1988–2011 for the BATS station, 1988–2010 for the ALOHA station, 1987–2006 for the OSP and 1995–2012 for the CARIACO station. Because sediment traps are deployed in the water for several months, their measurements represent an average for a relatively extended period instead of a snapshot. The results of a are based on the CbPM NPP product and an e-folding remineralization time of 12 h for labile DOC.

Source Data

Our advective-diffusive export is calculated by tracking subsurface organic carbon respiration rates back to the base of the euphotic zone using an adjoint method[31]. The semi-labile and labile organic carbon fluxes are 1.67 ± 0.02 and 2.70 ± 1.04 Pg C year−1, respectively, at 73.4 m. The export of refractory organic carbon (e-folding decay time about 5,500 and about 11,000 years in and below the euphotic zone, respectively) is two orders of magnitude lower than that of labile and semi-labile ones and, thus, is ignored in the following discussion. Our advective-diffusive flux of semi-labile organic carbon is close to a previous estimate of 1.8 Pg C year−1 at 100 m reference depth[19], but lower than the estimated 2.31 ± 0.6 Pg C year−1 at the same depth of 73.4 m obtained from interpolated DOC observations and a circulation model[20]. When we include the export of labile organic carbon, our estimate surpasses any previous estimates. The previous estimate[20], which considered only one DOC pool, may have included signals from both labile and semi-labile organic carbon, explaining its intermediate value.

The labile and semi-labile organic carbon have distinct export patterns (Fig. 3a,b and Extended Data Fig. 7). Two factors contribute to this spatial pattern. One is the biological production pattern and the other is the spatially variable export efficiency. To explain the latter effect, we computed the mean DOC sequestration time for each water column in the model (‘DOC sequestration time’ in Methods). The mean of these residence-time distribution functions is contoured in Fig. 3d. For semi-labile organic carbon, the high export regions are in the Southern Ocean convergence zone, subarctic North Pacific and North Atlantic, with relatively long DOC residence time (Fig. 3b,d). These are important subduction and deep-water formation regions in which water masses are transferred from the mixed layer into the thermocline and deep ocean. For labile organic carbon, the high export regions (Fig. 3a) are located in the periphery of where it is produced, for example, in the subtropical gyres. However, there is no apparent export in the equatorial oceans and coastal upwelling regions (for example, the Arabian Sea and eastern tropical Pacific), in which its production is the highest (Extended Data Fig. 6c,d). This is because strong upwelling retains the labile organic carbon in the surface ocean long enough for it to be respired. Another interesting region is the high-latitude North Atlantic Ocean, in which export is high even though production is low. This is because strong vertical mixing reinforces the export of short-lived organic carbon (Fig. 3d).

Fig. 3: Contour plots of advective-diffusive export flux at the base of the model euphotic zone.

a, Distribution of advective-diffusive flux by labile organic carbon (mg C m−2 day−1). b, Distribution of advective-diffusive flux by semi-labile organic carbon (mg C m−2 day−1). c, Distribution of the ratio of advective-diffusive flux to TOC flux. d, Distribution of DOC residence time in years at the bottom of the euphotic zone. The residence time is defined as the time elapsed for DOC to be upwelled to the surface ocean following its export below the euphotic zone at that grid box. The results are based on the CbPM NPP product and an e-folding remineralization time of 12 h for labile DOC.

Source Data

Regionally, the contribution of advective-diffusive export (labile + semi-labile) to total carbon export can be higher than 50% (ref. [4]) (Fig. 3c and Extended Data Fig. 7). The high-contribution regions are mainly in the middle-latitude and high-latitude oceans, such as the subtropical North Atlantic and South Atlantic oceans, and high-latitude North Atlantic Ocean and the Southern Ocean convergence zones, whereas in the equatorial upwelling zones, the contribution of advective-diffusive export is less than 10%. Overall, our estimated pattern of advective-diffusive flux is in close agreement with the results estimated on the basis of an inverse model constrained using the US Climate Variability and Predictability (CLIVAR) DOC observations[32]. The zonally averaged advective-diffusive export proportion (sum of advective-diffusive fluxes by labile and semi-labile organic carbon over TOC flux) increases from about 15% in equatorial regions (0–15°) to about 37%, about 39% and about 29% in subtropical (15–30°), temperate (30–45°) and subpolar (45–60°) areas, respectively. The poleward increase of advective-diffusive export ratios is consistent with the mechanisms of the mixed-layer pump17,33, eddy subduction pump[18] and large-scale subduction pump[34].

Combining the non-advective-diffusive and advective-diffusive fluxes, our globally integrated TOC flux at the base of the euphotic zone is 15.00 ± 1.12 Pg C year−1 (Fig. 4a). This number is sensitive to the export horizon owing to strong remineralization in the upper ocean. For example, the export flux decreases by roughly 30% from 73 m to the 100-m-depth horizon typically used by ESMs as a reference export depth. An alternative perspective on this sensitivity is provided by distribution functions for the sequestration time, τ, of organic carbon production and for the stock of regenerated DIC (Fig. 5). The TOC production with τ > 3 months is 11.09 ± 1.02 Pg C year−1. For τ > 1 year, the total export flux decreases to 8.25 ± 0.30 Pg C year−1 and for τ > 3 years, it is only 6.30 ± 0.09 Pg C year−1. The distribution functions show that the total flux is dominated by small residence-time export, but that the small residence-time fluxes contribute negligibly to the standing stock of regenerated DIC, pointing to the rapid recycling of much of the organic matter production on short timescales. For τ < 1 year (yellow regions in Fig. 5), the accuracy of export fluxes is highly uncertain as a result of three main factors. First, the circulation model lacks representation of the seasonal cycle. Second, the short residence-time fluxes are sensitive to the mathematical formulation of the biological production and respiration models. Last, the inverse model, which is constrained by carbon, oxygen and nutrient stocks, is insensitive to the part of the export-flux distribution that does not affect these stocks. Indeed, marked contributions to the standing stock (Fig. 5b) only become apparent when residence times approach approximately 1 year.

Fig. 4: TOC flux at the base of the model euphotic zone.

a, Distribution of TOC (non-advective-diffusive + advective-diffusive) flux (mg C m−2 day−1) at the base of the euphotic zone. b,c, Comparisons of TOC flux with geochemical ANCP estimates at the North Atlantic and Pacific oceans, respectively. The black squares represent the mean (±1σ) of TOC flux over 10° latitude bands in this study. The red triangles correspond to the mean (±1σ) of geochemical ANCP estimates[33], which is computed at the base of a spatially varying mMLD obtained from a CESM simulation. For a fair comparison, we extend our flux to the bottom of mMLD at places in which mMLD is deeper than the model euphotic zone depth (Methods). Our TOC flux is the sum of non-advective-diffusive and advective-diffusive flux at places in which mMLD is shallower than the euphotic depth. The results are based on the CbPM NPP product and an e-folding remineralization time of 12 h for labile DOC.

Source Data

Fig. 5: Sequestration-time distribution functions for the organic carbon flux and the stock of regenerated DIC.

a, Sequestration-time-partitioned organic carbon production. The curves show the cumulative NPP fluxes with sequestration times greater than τ separated into contributions from labile DOC (red), semi-labile DOC (black), refractory DOC (green) and POC (blue). The sequestration times are measured from the time when the organic carbon is respired into DIC to the time when the regenerated DIC is transported back to the 36.1-m-thick surface layer of the model. b, Sequestration-time-partitioned standing stock of regenerated DIC. The curves show the cumulative stock with sequestration times less than τ. All curves correspond to climatological-mean estimates integrated over the whole ocean volume. The error bars, indicated by the shaded regions, correspond to ±1σ computed from four inverse models in which the e-folding lifetime of labile DOC was either 12 h or 24 h and the biological carbon production was patterned using either the CbPM or the CAFE NPP products. The posterior parametric uncertainty makes a negligible contribution to the shown error bars. For τ > 1 year (green regions), the inverse model produces a robust estimate of the export-flux distribution.

Source Data

Our estimated TOC flux rate at 100 m (10.64 ± 0.80 Pg C year−1) falls into the range of the previous model and satellite-based predictions (5–12 Pg C year−1, summarized in ref. [14] in their table 1) and is in close agreement with the ‘baseline’ estimate of 10.2 Pg C year−1 using an ensemble numerical model constrained with O_2 and DOC observations[24]. There are no direct global-scale annual TOC flux measurements because extensive samplings would be needed to resolve the seasonal cycle of all export pathways. Reliable ANCP (equivalent to TOC flux at steady state) estimates are only available at time-series stations and some basins based on regional ARGO float data[35]. On basin scales, our results align with the geochemical ANCP in the Pacific Ocean and North Atlantic Ocean[35] (Fig. 4b,c). The magnitude of our estimated TOC export flux varies meridionally by approximately a factor of three, indicating a smaller gradient compared with previous ESM-based or satellite-based estimates, which typically suggest that TOC export varies by up to a factor of ten[35]. We further compared our TOC flux with those measured using mass-balance calculations at ALOHA, BATS and OSP. Our model results (mean with ±σ) at the base of maximum mixed-layer depth (mMLD) have overlapping error bars with mass-balance estimates at ALOHA (45.99 ± 23.00 this study versus 82.15 ± 23.00 mg C m−2 day−1) and at OSP (52.65 ± 3.29 this study versus 75.56 ± 19.71 mg C m−2 day−1)[14]. Our estimate at the BATS station (23.00 ± 3.28 mg C m−2 day−1 at mMLD) is much lower than the ANCP by Emerson[14] (124.83 ± 39.42 mg C m−2 day−1 at 150 m), but twofold higher than the ANCP determined using O_2 and DI^13C in the western North Atlantic around the BATS station at 100 m depth (82.13 ± 13.14 this study versus 39.42 mg C m−2 day−1 (ref. [36])).

Biogeochemical implications

Budgets based on in situ observations often struggle to establish a balance between community production and respiration (for example, refs. 37,38), either because they fail to account for all processes that deliver organic carbon to the mesopelagic ocean or because they are limited to measurements during a specific season. Our model, which represents an annual-mean balance between community production and respiration, is able to simultaneously fit full water-column observations of DIC, DOC, ALK and O_2, showing that there is no difficulty in closing the budget provided one accounts for both advective-diffusive and non-advective-diffusive export pathways. At the Porcupine Abyssal Plain site in the North Atlantic Ocean, our TOC export (201.5 ± 29.4 mg C m−2 day−1 between 73 and 1,000 m) exceeds the in situ community respiration (48–167 mg C m−2 day−1 between 50 and 1,000 m) measured in the summer season[39] when net community production is relatively low[38]. At station ALOHA, our annual TOC flux between mMLD and 1,000 m (45.1 ± 4.0 mg C m−2 day−1) overlaps with in situ measurements of heterotrophic respiration rates between 150 and 1,000 m (32.5–96.6 mg C m−2 day−1)[37]. However, at the Japanese time-series site K2 station, also in the Pacific, our TOC flux between mMLD and 1,000 m (82.1 ± 2.4 mg C m−2 day−1) falls short of the lower end of in situ determinations (106.1–249.8 mg C m−2 day−1)[37] at the depth interval of 150–1,000 m. Such disparities could potentially arise because our model represents an annual mean, whereas the in situ measurements were conducted during specific seasons. Future development of a seasonal inverse model could contribute to narrowing this difference. The disparities might also be influenced by the inherent uncertainties associated with in situ measurements. In light of these potential factors, we advocate for an increased number of in situ observations focused on year-round whole-community carbon demand within the twilight zone.

Numerous mechanisms have been proposed to explain the spatial variations of carbon flux, with prominent factors including particle size and sinking velocities, community structure, remineralization dependence on temperature and oxygen, and ballast effect40,41. ESMs that incorporate these mechanisms in varying degrees exhibit a wide range of carbon flux (approximately 5–12 Pg C year−1)[3] and have clearly identifiable biases in their simulated oxygen and carbon distributions. By contrast, our inverse model avoids overparameterization, by not including explicit representations of each of these processes. Nevertheless, it provides a good fit to the tracer data with a simple temperature-dependent parameterization for the remineralization of organic carbon. Specifically, our model adopts a power-law parameterization with a temperature-dependent exponent b = bCθT + bC for non-advective-diffusive carbon fluxes (Methods). Our inversion infers a temperature dependence, bCθ = 0.03 °C−1 (Extended Data Table 1) that is approximately 50% smaller than the value estimated using a limited sediment trap dataset of POC fluxes[42], but is otherwise in agreement with the sign of the temperature effect. Geographically, non-advective-diffusive vertical fluxes attenuate faster when surface waters are warmer and penetrate deeper in the water column when surface waters are cold (Extended Data Fig. 8). Notably, our non-advective-diffusive vertical flux includes not only the classical gravitational POC flux but also any fluxes with substantial non-advective-diffusive vertical transport, such as fluxes related to seasonal lipid pump[22] and zooplankton migration pump[21]. It is also worth noting that, in high-latitude low-temperature oceans, the prevalence of large phytoplankton with ballast shells and shorter food webs promotes non-advective-diffusive vertical fluxes. Conversely, in warm subtropical gyres, the prevalence of small phytoplankton and longer food webs reduces non-advective-diffusive flux40,41. The deeper penetration in higher latitudes, coupled with an overall lower temperature dependence compared with the trap-derived value (0.03 °C−1 this study versus 0.062 °C−1 (ref. [42])) underscores the intricate interplay of different mechanisms. In our inverse model, the dependence of the power-law exponent on temperature serves as a proxy for any mechanism that correlates with surface temperature. Future research will need to unravel these mechanisms. But if we assume that the contemporary relationships persist into the future, we can expect that global warming will cause stronger non-advective-diffusive vertical-flux attenuation (increased b-value in Extended Data Fig. 8c,d), which would leave more carbon in the upper ocean and atmosphere[43]. The same mechanism could help to explain atmospheric CO_2 variations during glacial–interglacial cycles[44]. The more efficient downward carbon transfer in cold waters compared with warm waters (evidenced by lower b-value in high latitudes; Extended Data Fig. 8) suggests a stronger removal of CO_2 from the atmosphere during cold climates.

Our results emphasize the role played by advective-diffusive export. Only a few global-data-constrained estimates of carbon export23,24 and algorithms account for advective-diffusive export of DOC and suspended POC (refs. 7,45,46) or export mediated by zooplankton migration[3]. Previously, the contribution from DOC was typically included by simply scaling up the POC flux by an assumed amount[2]. However, Emerson[14] found that sinking POC export is a small fraction of the ANCP at three time-series stations (BATS, ALOHA and OSP), suggesting that other export pathways are important. Indeed, we find that the export of DOC and suspended POC can be regionally important, especially in subtropical gyres in which DOC production is high and Ekman convergence transports DOC downward4,20 and in high-latitude oceans in which the subduction pump and mixed-layer pump are strong17,18 (Fig. 3c). More importantly, in situ observations often miss such mixing events because sea-going measurements usually take place during the summer, when there is less vertical mixing in the water column. This is a possible reason why POC export ratios determined in situ are negatively correlated with NPP in the Southern Ocean[10]. Indeed, we find that up to 70% of the production is exported by means of the advective-diffusive pathway in the latitudes between the subtropical and subantarctic fronts (Fig. 3c). The negative correlation between POC export ratio and NPP contradicts the empirical relationships that relate the ef-ratio to temperature and NPP (refs. 6,45) by assuming a positive relationship between NPP and the ef-ratio.

Furthermore, the export of DOC is not associated with the export of particulate inorganic carbon (PIC) as the POC export may be. Such export can therefore be more efficient at sequestering CO_2 by avoiding the effects of the carbonate counter pump[47]. However, a more sluggish circulation48,49 and stronger stratification[50] expected as a result of future warming may decrease the export of DOC and suspended POC and thus contribute a positive feedback to climate warming. An improved mechanistic understanding of the various pathways associated with the BCP should help to decipher what controls carbon export efficiency and improve predictions of future carbon exports11,12,13. Our results highlight the importance of including the advective-diffusive flux of DOC and suspended POC when estimating the strength of the BCP and motivate the need to improve satellite-based carbon export algorithms so that they better account for export mediated by mixing and other fluid transport.

One strength of our inverse model is that the estimated export fluxes are not sensitive to satellite-estimated NPP. This is a substantial difference from export estimates based on the ef-ratio, which suffer from the compound uncertainties in the ef-ratio and in the algorithm used to estimate the NPP (ref. [28]). By contrast, our inverse model infers carbon export from the respiration signal imprinted in the full water column DIC, DOC, DIP, ALK and oxygen observations. Unlike prognostic ESMs, our top-down inverse estimate avoids the need for incorporating uncertain and possibly incorrect parameterizations of complex processes for which we have insufficient understanding. However, our model has its own limitations. For example, our advection–diffusion transport model represents the climatological annual-mean circulation and lacks seasonality. Therefore, we are unable to diagnose how export changes seasonally. Future developments of our inverse model should consider the effect of seasonal variation. Finally, the successful integration of DIC and oxygen measurements in our model was contingent on an accurate estimation of the transient anthropogenic carbon signal. Our estimate shows that the vertical DIC gradient in the ocean has decreased by approximately 20% owing to the invasion of anthropogenic CO_2 (Extended Data Fig. 5c and ‘Anthropogenic DIC’ in Methods). We therefore expect that future improvements in anthropogenic carbon-uptake estimates will need to take into account the multitracer constraints we used here.

Methods

Data

Observational concentrations of DIP, DIC, ALK and O_2 were downloaded from the Global Ocean Data Analysis Project website, the second version (GLODAPv2 (ref. [15]); Extended Data Fig. 1). DOC observations were from ref. [16]. The data were then binned into the ocean circulation inverse model (OCIM) model grid that has a horizontal resolution of 2° × 2° and vertically 24 layers, with finer resolution in the upper ocean and coarser resolution in the deep ocean. The updated DOC compilation contains 25,869 valid data points (Extended Data Fig. 2) after binning to our model grid and has better coverage than previously widely used ones (ref. [51]) that only had 14,034 valid data points in the model grid. The DOC dataset has a slight seasonal bias, with more samples collected in the summer season. However, we think that the influence is minor because: (1) a substantial proportion of the total DOC is composed of refractory DOC; unlike labile and semi-labile DOC, refractory DOC does not exhibit strong seasonality owing to its long residence time in the ocean; (2) we used tracer data from the full-water depth to constrain our model parameters. The deep ocean experiences lesser seasonal variability compared with the surface ocean. Therefore, using full-water-depth data helps to anchor the stability of the inversion. Two NPP products, carbon-based NPP from Sea-viewing Wide Field-of-view Sensor (SeaWiFS CbPM)[52] and CAFE, were downloaded from https://doi.org/10.6084/m9.figshare.19074521. The NPP products were interpolated and averaged by Nowicki et al.[24] to the same model grid as used in this study. The climatological ocean temperature and silicate are from World Ocean Atlas 2018 (refs. 53,54). The projected temperature at 2099 was obtained from a CESM-BGC model prediction under the RCP8.5 scenario[55]. The historical atmospheric pCO_2 data were obtained from ref. [56] for the period from 1850 to 2015 and were downloaded from https://scrippsco2.ucsd.edu/data/atmospheric_co2/primary_mlo_co2_record.html (ref. [57]) for the period from 2016 to 2020.

Biogeochemical inverse model

A schematic of the structure of the biogeochemical model is shown in Fig. 1. The model couples the cycling of phosphorus (P), carbon (C) and oxygen (O). The phosphorus model is the base model that provides a biological uptake rate (γ(r), in which r is a position coordinate) in P units (G ≡ (γ[DIP])), which is then converted to a DIC uptake rate in the carbon model by incorporating a C:P ratio (rC:P). In the P-cycle model, the DIP assimilation rate is modelled using a spatial pattern obtained from satellite-derived NPP (mg C m−2 day−1) and a gridded surface DIP climatology as follows

$$\gamma \left({\bf{r}}\right)\equiv \left\{\begin{array}{c}\alpha \frac{{\left[\frac{1}{{r}_{{\rm{C:P}}}}\frac{{\rm{NPP}}({\bf{r}})}{{{\rm{NPP}}}_{0}}\right]}^{\beta }}{\frac{{\left[{\rm{DIP}}\right]}_{{\rm{obs}}}({\bf{r}})}{{[{\rm{DIP}}]}_{0}}},\,{\rm{if}}\,z < {z}_{{\rm{c}}},\\ 0,\,{\rm{otherwise}}\end{array}\right.$$

(1)

in which NPP_0 and [DIP]0 are 1 mmol C m−2 day−1 and 1 μM that are functioned to remove dimensions of NPP and DIP; α and β are adjustable parameters that are constrained in the inversion; rC:P is the C:P ratio that is used to convert NPP from C unit to P unit and modelled according to Galbraith and Martiny[58] (rC:P = (0.006 + 0.0069[DIP]obs)−1). z and zC are water depth and the euphotic zone depth, respectively. Photosynthesis is assumed to occur only in the euphotic zone and to be zero below. The euphotic zone is defined as the top two model layers (73.4 m).

Phosphorus model

The phosphorus model considers four explicit tracers: dissolved inorganic phosphorus (DIP), dissolved semi-labile organic phosphorus (DOP), dissolved labile organic phosphorus (DOPl) and particulate organic phosphorus (POP). We assign an e-folding remineralization time (1/κl) of 12 h for DOPl so that it quickly cycles in the upper ocean, with little being transported below the euphotic zone. We use a parameter δ to allocate production to labile pools. The remaining production (total production less production to DOPl) is allocated to DOP and POP. The factions σP and (1 − σP − δ) of the production allocated, respectively, to DOP and POP are determined by estimating the parameter σP through our Bayesian inversion procedure. The advective-diffusive transport of dissolved tracers (DIP, DOP and DOPl in the P model; DIC, semi-labile dissolved organic carbon (DOC), labile dissolved organic carbon (DOCl), refractory dissolved organic carbon (DOCr) and ALK in the C model; and O_2 in the O model) is computed using the OCIM tracer transport matrix, \({\bf{T}}\left[{\rm{C}}\right]\equiv \nabla \cdot \left(\vec{U}\left[{\rm{C}}\right]-{\rm{K}}\nabla \left[{\rm{C}}\right]\right)\), in which \(\vec{U}\) is the velocity vector and K is the diffusive term. Τ represents the climatological mean circulation of the ocean. The OCIM tracer transport matrix is constrained using salinity, temperature, sea-surface height, CFC-11, CFC-12, 14C, 3He etc. (see DeVries and Holzer[59] for details). We neglect the advective-diffusive transport of particulate tracers (POP in the P model and PIC and POC in the C model) so that particulate tracers are transported only vertically. The vertical transport of POP is modelled using a sinking flux divergence operator (\({{\bf{F}}}_{{\rm{POP}}}\equiv \nabla \cdot (\vec{w}[{\rm{POP}}])\)), in which \(\vec{w}\) is the sinking speed of POP and is directed downward. We choose a sinking speed that increases linearly with depth and a constant dissolution rate, κP = (1/30) days−1, so that the attenuation of the vertical flux of POP follows a power-law function, F(z) = F(z_0)(z/z_0)−b, in which F(z) and F(z_0) are fluxes at a depth of z and z_0, respectively[60]. A sensitivity test with κP = (1/60) days−1 suggests that the choice of κP does not markedly influence our results. The exponent b for the P model (C model in the following section) is defined in the following way (ref. [42]), b(P) = bPθT + bP, in which bPθ and bP are two adjustable parameters and T is the average temperature of the model euphotic zone. The initial guess of bPθ is set to zero, thereby avoiding any intentional imposition of temperature dependence. The optimization process determines both the sign and magnitude of bPθ. The governing equations for the phosphorus cycle are as follows:

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right][{\rm{DIP}}]=-\,\gamma [{\rm{DIP}}]+{\kappa }_{{\rm{p}}}[{\rm{POP}}]+{\kappa }_{{\rm{dP}}}[{\rm{DOP}}]+{\kappa }_{{\rm{l}}}[{{\rm{DOP}}}_{{\rm{l}}}]+{\kappa }_{{\rm{g}}}([{\rm{DIP}}]-{\overline{[{\rm{DIP}}]}}_{{\rm{obs}}}),$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right][{\rm{DOP}}]={\sigma }_{{\rm{P}}}\gamma [{\rm{DIP}}]-{\kappa }_{{\rm{dP}}}[{\rm{DOP}}],$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right][{{\rm{DOP}}}_{{\rm{l}}}]=\delta \gamma [{\rm{DIP}}]-{\kappa }_{{\rm{l}}}[{{\rm{DOP}}}_{{\rm{l}}}],$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{{\bf{F}}}_{{\rm{POP}}}\right][{\rm{POP}}]=(1-{\sigma }_{{\rm{P}}}-\delta )\gamma [{\rm{DIP}}]-{\kappa }_{{\rm{P}}}[{\rm{POP}}],$$

(2)

in which κdP is the DOP remineralization rate constant that is a function of temperature defined using a Q_10 function (κdP = κPθQ_10(T−30)/10), in which T is water temperature from World Ocean Atlas 2018 (ref. [54]). κPθ and Q_10 are optimized in the inversion. κl is the e-folding remineralization time of DOPl, which is fixed at κl = (1/12) h−1. We tested the sensitivity to a smaller κl = (1/24) h−1 and found that the choice of κl did not substantially change the fittings to the tracers but could alter the export flux of labile organic matter. We therefore include different κl values in the uncertainty analysis (see the ‘Uncertainty analysis’ section). κg is prescribed to (1/10^6) years−1 and is used to set the global mean DIP concentration to the observed global mean concentration ([DIP]obs). κP is a prescribed POP remineralization rate constant (κP = (1/30) days−1). A sensitivity test shows that increases or decreases in the fraction of DOPl production (δ in equations (2) and (3)) does not alter the fit to the observational data nor does it change the inferred export fluxes of POC and semi-labile DOC. We therefore set δ to be zero in the first-round optimization.

Carbon model

The carbon model explicitly simulates seven tracers: DIC, DOCl, DOC, DOCr, POC, PIC and ALK (Fig. 1b). The DIP assimilation rate G is converted to the DIC assimilation rate by incorporating a C:P ratio (rC:P) that is allowed to vary spatially according to the modelled DIP concentration, rC:P = (cc[DIP] + dd)−1, in which cc and dd are estimated as part of the inversion. As in the P model, we set the allocation to DOCl to be zero in the first round of optimization. Subsequently, we prescribe the difference between satellite NPP and model organic carbon production as the production for labile DOCl, so that our model production matches satellite NPP exactly. The fraction, σC, of the organic carbon production allocated to POC and DOC pools is estimated as part of the inversion and does not need to be the same as the fraction σP allocated to the POP and DOP pools. A further adjustable parameter, η, is used to control the fraction of DOC that is transferred to the refractory pool by bacterial reworking. The remaining DOC fraction (1 − η) is remineralized back to DIC. The e-folding decay times of DOCr (κur and κdr for the upper and deeper ocean, respectively) are estimated as part of the inversion. POC sinks and is gradually remineralized to DIC in the water column. The downward transport of POC is modelled using a flux divergence operator (FPOC), which is formulated in the same way as the POP sinking flux-divergence operator FPOP with independent adjustable parameters bCθ and bC that are determined as part of the inversion (Extended Data Table 1). Unlike DIP, DIC experiences sea-to-air gas exchange at the surface. This gas exchange is modelled according to the method used for phase 2 of the Ocean Carbon-Cycle Model Intercomparison Project (OCMIP-2)[61] using a recalibrated piston velocity (see the next section). Also, freshwater precipitation and evaporation can greatly affect surface ocean DIC and ALK concentrations. Precipitation will dilute, whereas evaporation will concentrate their concentrations. A virtual flux according to OCMIP-2 (ref. [61]) is applied to model for the effects of precipitation and evaporation on DIC and ALK (FvDIC[DIC]s and FvALK[ALK]s, in which [DIC]s and [ALK]s are the mean surface-ocean concentrations of DIC and ALK, respectively).

Production of PIC is modelled to be proportional to the production of POC using two adjustable parameters, rSi and rRR, that are estimated in the inversion. The parameter rSi adjusts PIC production according to silicate concentration in the surface ocean in linear form (RRR = rSi[SiO_44−] + rRR). The downward transport of PIC is modelled using a flux divergence operator (FPIC), which generates a PIC flux profile that follows an exponential function FPIC(z) = F_0exp((z − z_0)/d), in which d is the PIC dissolution length scale, whose value is estimated as part of the inversion (Extended Data Table 1). Compared with a power-law function, an exponential function with a length scale on the order of several thousand metres leads to a much smaller CaCO_3 dissolution rate in the shallow water in which CaCO_3 is supersaturated[62]. Every mole of PIC production consumes two moles of ALK. By contrast, the dissolution of one mole of PIC releases two moles of ALK (equation (3)). From the perspective of carbon, photosynthesis and remineralization of organic matter do not change alkalinity. However, in the processes of photosynthesis and remineralization, chemical forms of nitrogen change, which influences alkalinity so that a mole of organic carbon production increases alkalinity by rN:C moles, whereas a mole of organic carbon remineralization decreases alkalinity by rN:C moles. The governing equations for carbon cycling are as follows:

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right][{\rm{D}}{\rm{I}}{\rm{C}}]=-({\bf{I}}+(1-{{\rm{\sigma }}}_{{\rm{C}}}-\delta ){r}_{{\rm{R}}{\rm{R}}}){\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}+\eta {\kappa }_{{\rm{d}}{\rm{C}}}[{\rm{D}}{\rm{O}}{\rm{C}}]+{\kappa }_{{\rm{l}}}[{\rm{D}}{\rm{O}}{{\rm{C}}}_{{\rm{l}}}]+{\kappa }_{{\rm{r}}}[{\rm{D}}{\rm{O}}{{\rm{C}}}_{{\rm{r}}}]+{\kappa }_{{\rm{P}}{\rm{I}}{\rm{C}}}[{\rm{P}}{\rm{I}}{\rm{C}}]+{\kappa }_{{\rm{p}}}[{\rm{P}}{\rm{O}}{\rm{C}}]+{{\bf{F}}}_{{{\rm{C}}{\rm{O}}}_{2}}+{{\bf{F}}}_{{\rm{v}}{\rm{D}}{\rm{I}}{\rm{C}}}{[{\rm{D}}{\rm{I}}{\rm{C}}]}_{{\rm{s}}},$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right]\left[{\rm{DOC}}\right]={\sigma }_{{\rm{C}}}{\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}-{\eta \kappa }_{{\rm{dC}}}\left[{\rm{DOC}}\right],$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{{\bf{F}}}_{{\rm{POC}}}\right]\left[{\rm{POC}}\right]=\left(1-{\sigma }_{{\rm{C}}}-\delta \right){\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}-{\kappa }_{{\rm{p}}}\left[{\rm{POC}}\right],$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{{\bf{F}}}_{{\rm{PIC}}}\right]\left[{\rm{PIC}}\right]=\left(1-{\sigma }_{{\rm{C}}}-\delta \right){R}_{{\rm{RR}}}{\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}-{\kappa }_{{\rm{PIC}}}\left[{\rm{PIC}}\right],$$

$$\begin{array}{l}\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right][{\rm{ALK}}]=-2(1-{\sigma }_{{\rm{C}}}-\delta ){R}_{{\rm{RR}}}{\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}+{r}_{{\rm{N}}:{\rm{C}}}{\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}\\ \,\,\,\,-{r}_{{\rm{N}}:{\rm{C}}}(\eta {\kappa }_{{\rm{dC}}}[{\rm{DOC}}]+{\kappa }_{{\rm{r}}}[{\rm{DO}}{{\rm{C}}}_{{\rm{r}}}]+{\kappa }_{{\rm{l}}}[{\rm{DO}}{{\rm{C}}}_{{\rm{l}}}]+{\kappa }_{{\rm{p}}}[{\rm{POC}}])\\ \,\,\,\,\,+2{\kappa }_{{\rm{PIC}}}[{\rm{PIC}}]-{{\bf{F}}}_{{\rm{vALK}}}{[{\rm{ALK}}]}_{{\rm{s}}}+{\kappa }_{{\rm{g}}}([{\rm{ALK}}]-{\overline{[{\rm{ALK}}]}}_{{\rm{obs}}}),\end{array}$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right]\left[{\rm{DO}}{{\rm{C}}}_{{\rm{l}}}\right]=\delta {\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}-{\kappa }_{{\rm{l}}}\left[{\rm{DOC}}\right],$$

$$\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right]\left[{\rm{DO}}{{\rm{C}}}_{{\rm{r}}}\right]=\left(1-\eta \right){\kappa }_{{\rm{dC}}}\left[{\rm{DOC}}\right]-{\kappa }_{{\rm{r}}}\left[{\rm{DO}}{{\rm{C}}}_{{\rm{r}}}\right].$$

(3)

Anthropogenic DIC

To use DIC observations to constrain our inverse model, we have to take into account the changing DIC concentration owing to the invasion of anthropogenic CO_2 into the ocean. To obtain a self-consistent estimate of the anthropogenic carbon signal, we performed a time-dependent simulation using equation (3). Starting from an assumed steady state, we time-stepped our carbon-cycle model forward in time from 1850 to 2020, using an implicit trapezoid-rule time-integration scheme for all terms except for the gas exchange, for which we used an explicit Euler forward scheme. In this calculation, we prescribed the surface SST according to a time-dependent reanalysis product (ref. [63]). The transient integration was carried out with a time step size of Δt = 2 months. The atmospheric pCO_2 was prescribed according to ref. [56] from 1850 to 2015 and according to ref. [57] from 2016 to 2020. We also simulated δ^14C to better calibrate the air–sea gas-exchange velocity as described below. The atmospheric δ^14C was prescribed according to ref. [64] for the period from 1850 to 2015 and according to ref. [65] from 2016 to 2020. To produce the initial conditions, we assumed that the system was in steady state in 1850 and used Newton’s method to find the steady state.

To calibrate the air–sea gas exchange parameterization, we re-optimized the scaling factor in the OCIM2 gas-exchange scheme by minimizing the misfit between our modelled δ^14C and the GLODAPv2 δ^14C data. See Extended Data Fig. 5a,b for the number of observations as a function of time. To compute the misfit, we sampled our model at the location and times of the bottle measurements in the GLODAPv2 database. Our calibration method followed an iterative two-step process in which we first optimized the air–sea gas exchange through a series of transient carbon-cycle simulations. After obtaining the optimal air–sea gas exchange, we subtracted the excess anthropogenic DIC from the GLODAPv2 measurements to produce an estimate of the natural background DIC for the year 1850. The resulting DIC data and optimal gas-exchange velocity were then used for the optimization of the biogeochemistry model (see the ‘Parameter estimation’ section). The optimized biogeochemical model was then used to produce an updated initial condition for the transient carbon-cycle simulation and a re-optimization of the air–sea gas-exchange velocity. We repeated this two-step process until we obtained self-consistent estimates of: (1) the optimal biogeochemical parameter values (Extended Data Table 1); (2) the biogeochemical state; (3) the scaling factor for the air–sea gas–transfer velocity, a = 0.234 cm h−1 (m s−1)−2; (4) transient DIC; and (5) the transient δ^14C signal including the combined effects of radioactive decay, the Suess effect and the bomb radiocarbon signal. Extended Data Figure 5c shows a time series of the excess anthropogenic DIC concentration averaged over the top 100 m of the water column and for the water column below 100 m. By 2020, the vertical DIC gradient is reduced by 20%.

Oxygen model

Oxygen production is modelled by applying a ratio of oxygen to carbon (rO:C) to the DIC assimilation rate (GrC:P). The ratio rO:C is optimized in the process of inversion. We convert the DOC and POC remineralization rates (ηκdC[DOC] + κr[DOCr] + κl[DOCl] + κp[POC]) to an oxygen consumption rate using the same rO:C ratio and gradually shut down oxygen consumption as the oxygen concentration falls below the critical value (Ocrit = 5 mmol l−1) using a hyperbolic equation (R([O_2]) = 0.5 + 0.5tanh[([O_2] − Ocrit)/[O_2]0]), in which [O_2]0 (1 mmol l−1) is used to remove the O_2 dimension. Sea-to-air O_2 flux (FO2) is modelled according to OCMIP-2 (ref. [61]):

$$\begin{array}{l}\left[\frac{{\rm{d}}}{{\rm{d}}t}+{\bf{T}}\right][{{\rm{O}}}_{2}]={r}_{{\rm{O}}:{\rm{C}}}{\bf{G}}{r}_{{\rm{C}}:{\rm{P}}}+{{\bf{F}}}_{{\rm{O}}2}-{r}_{{\rm{O}}:{\rm{C}}}{\bf{R}}(\eta {\kappa }_{{\rm{d}}{\rm{C}}}[{\rm{D}}{\rm{O}}{\rm{C}}]\\ \,\,\,\,\,\,\,\,\,+{\kappa }_{{\rm{r}}}[{\rm{D}}{\rm{O}}{{\rm{C}}}_{{\rm{r}}}]+{\kappa }_{{\rm{l}}}[{\rm{D}}{\rm{O}}{{\rm{C}}}_{{\rm{l}}}]+{\kappa }_{{\rm{p}}}[{\rm{P}}{\rm{O}}{\rm{C}}])\end{array}$$

(4)

in which the matrix R is a diagonal matrix, whose elements are given by R([O_2]).

Parameter estimation

The 21 adjustable parameters of the model (Extended Data Table 1) were estimated using a Bayesian inversion method. In this approach, the solutions to our model equations define the tracer fields as implicit functions of the adjustable parameters, which we then compare with the observations to construct a likelihood function. We obtain the P, C and O fields by finding the steady-state solutions of the governing equations for the P, C and O models (equations (2)–(4)). Because the governing equations for the P model are linear, their steady-state solution can be obtained efficiently by direct matrix inversion after setting the time derivatives in equation (2) to zero. We fix the atmospheric CO_2 concentration at the preindustrial level (278 ppm) to compute the preindustrial sea-to-air CO_2 flux (FCO2). The steady-state solution for the C model is solved using Newton’s method because of nonlinearity in FCO2. The governing equation for O is also nonlinear because of the hyperbolic function (R) that turns off oxygen consumption when oxygen concentration is critically low. We solve the oxygen equations using Newton’s method.

To find the most probable parameter values, we minimize the negative logarithm of the posterior probability function, which is equivalent to minimizing the negative log-likelihood because we log-transformed our parameters (except of the slopes of exponent b) so that they have flat priors:

$$f=\frac{1}{2}\left({{e}_{{\rm{DIP}}}^{{\prime} }{\bf{W}}}_{{\rm{P}}}{e}_{{\rm{DIP}}}+{e}_{{\rm{DIC}}}^{{\prime} }{{\bf{W}}}_{{\rm{DIC}}}{e}_{{\rm{DIC}}}+{e}_{{\rm{DOC}}}^{{\prime} }{{\bf{W}}}_{{\rm{DOC}}}{e}_{{\rm{DOC}}}+{e}_{{\rm{ALK}}}^{{\prime} }{{\bf{W}}}_{{\rm{ALK}}}{e}_{{\rm{ALK}}}+{e}_{{{\rm{O}}}_{2}}^{{\prime} }{{\bf{W}}}_{{{\rm{O}}}_{2}}{e}_{{{\rm{O}}}_{2}}\right)+{\rm{const.}},$$

(5)

in which the eX are column vectors whose elements are given by the difference between modelled and observed concentrations, eX = HX[Xmod] − [Xobs], in which the X label denotes the specific tracer and Hx is a rectangular matrix that picks out the model grid boxes that have observations of tracer X. Because there are no measurements that precisely separate DOC into different pools according to their lability, we sum all three pools in the model (DOC, DOCl and DOCr) and compare the sum to observations. For DIC, we subtracted our estimated anthropogenic DIC from the bottle measurements in the GLODAPv2 database according to the location and time of measurement (see the ‘Anthropogenic DIC’ section). WX is a precision matrix for tracer X and is defined in the following way:

$${{\bf{W}}}_{{\rm{X}}}=\frac{1}{{\sigma }_{{\rm{X}}}^{2}}{{\bf{V}}}_{{\rm{X}}},$$

(6)

in which VX is a diagonal matrix with the fractional volumes of the model grid boxes (V = diag(ΔVi/ΣiΔVi), in which the subscript i is the index of the grid boxes that have at least one observation) and \({\sigma }_{{\rm{x}}}^{2}\) is the spatial variance of the observations, that is,

$${\sigma }_{{\rm{X}}}^{2}=([{{\rm{X}}}_{{\rm{obs}}}]-{\mu }_{{\rm{X}}}{)}^{{\prime} }\,{{\bf{V}}}_{{\rm{X}}}([{{\rm{X}}}_{{\rm{obs}}}]-{\mu }_{{\rm{X}}}),$$

(7)

with the spatial mean given by

$${\mu }_{X}=\frac{{{\bf{1}}}^{{\prime} }{{\bf{V}}}_{{\rm{X}}}[{{\rm{X}}}_{{\rm{obs}}}]}{{{{\bf{1}}}^{{\prime} }{\bf{V}}}_{{\rm{X}}}},$$

(8)

in which VX is a diagonal matrix with the grid-box volumes and the subscript X represents the grid boxes that have observations of tracer X. The bold 1 represents a column vector. The transpose turns it into a row vector. Thus, the numerator yields the volume integral of Xobs and the denominator yields the total volume.

The optimization is conducted using MATLAB’s fminunc function, which is computationally efficient because we can supply hand-coded first and second derivatives of the objective function with respect to the adjustable parameters. The optimization generally takes fewer than 100 iterations. The most probable model parameter values are presented in Extended Data Table 1. Parameter error bars that correspond to ±1 standard deviations are calculated using Laplace’s approximation as described in ref. [66].

Calculation of carbon flux

The two-dimensional vertical-flux field (fPOC) is calculated by vertically integrating POC remineralization below the euphotic zone (\({f}_{{\rm{P}}{\rm{O}}{\rm{C}}}={\sum }_{i=1}^{z}{\kappa }_{{\rm{p}}}{\rm{P}}{\rm{O}}{{\rm{C}}}_{i}{\Delta V}_{i}{M}_{i}\), in which i represents the index for deep grid, ΔVi is the volume of the ith grid box and Mi is a mask that is set to 1 below the euphotic zone and 0 elsewhere). In our model, POC is transported vertically in the water column and is not advected to neighbouring grids. This approximation is appropriate for the coarse horizontal resolution of our model. The non-advective-diffusive flux below the first two layers is calculated on the basis of the following power-law function (also known as the Martin curve function), fPOC(z) = fPOC(z_0)(z/z_0)−b, in which z_0 is the euphotic zone depth and z is the depth at which non-advective-diffusive flux is calculated. The exponent, b, depends linearly on the surface water temperature[42]. The estimated slope and intercept (bC and bCθ) for this linear relationship are presented in Extended Data Table 1. The advective-diffusive fluxes of labile and semi-labile organic carbon are calculated using an adjoint method, which tracks the export and subsequent remineralization of DOC, as described by ref. [31]. Only DOC respired below the depth of the euphotic zone is counted as export. The flux of TOC is the sum of the non-advective-diffusive flux and fluxes from labile DOC and semi-labile DOC. We ignore the export of refractory DOC because of its negligible contribution.

To compare the non-advective-diffusive flux to CMIP6 models at their consensus reference depth (about 100 m), we scale our estimated non-advective-diffusive flux using a power-law function with our optimized temperature-dependent b exponents.

To compare our export estimates to the geochemical ANCP estimates that are calculated at the base of spatially varying mMLDs obtained from a CESM simulation[35], we estimated the export fluxes of POC and semi-labile DOC to the depth z = mMLD. For cases in which mMLD is deeper than our euphotic zone depth, we scaled the fluxes using the power-law function with our optimized b exponents. For cases in which mMLD is above the base of the euphotic zone, we did not apply the power-law scaling because it tends to amplify errors. In those cases, we used the export flux at the base of the model’s euphotic zone for the comparison. Note that the contribution of labile DOC is ignored when scaling flux down to mMLD owing to the short e-folding decay time (12 h or 24 h).

Uncertainty analysis

The uncertainty analysis is conducted in two ways. First, we use a Monte Carlo method whereby an ensemble of parameter values is drawn from a multivariate normal distribution whose mean is given by our estimated most probable parameter values and whose covariance matrix is given by the inverse of the matrix of second partial derivatives of the negative logarithm of the posterior probability distribution, that is, by the Hessian matrix. For each ensemble member, we solve the steady-state model equations and calculate the organic carbon fluxes. However, the parameters are so well constrained that their uncertainties are small, and the flux uncertainties calculated this way are small. Second, because the DIP uptake model is constructed with two different satellite NPP products (SeaWiFS CbPM and CAFE), and the 21 adjustable parameters are optimized for each NPP field (Extended Data Table 1), the influence of NPP fields on export fluxes are much larger than that of parameter uncertainties. Also, the e-folding remineralization time of labile DOC is prescribed at 12 h and 24 h. We, therefore, report flux uncertainties estimated from the results based on different initial NPP fields and on different labile DOC e-folding decay timescales. The distributions of the standard deviation of key outputs are illustrated in Extended Data Fig. 9.

DOC sequestration time

To calculate the DOC sequestration time, we injected unit DOC pulses in the model euphotic zone and tracked this DOC as it was transported by the circulation, respired into DIC (according to the timescale given in Extended Data Table 1) and then transported back to the surface, where it was rapidly removed with a loss frequency of (1 day)−1. We then spatially integrated the removal rate for each DOC pulse to obtain residence-time distributions for the DOC exported from the surface of each water column.

Sequestration-time-partitioned distribution functions

To compute the sequestration-time-partitioned distribution functions, we use the three-dimensional organic carbon respiration rate to construct a Dirac δ-function pulse of labelled regenerated inorganic carbon. The resulting tracer field is then transported using the circulation model until it is removed in the 36.1-m-thick surface layer of the model using a loss frequency of (1/500) year−1. We integrate the system forward in time for 10,000 years, by which time all of the regenerated-carbon pulse has left the system. We use a second-order-accurate trapezoidal integration rule starting with a time-step size of less than 10−4 years and gradually increase it to 10 years by the end of the simulation. A sequestration-time density distribution function is obtained by globally integrating the loss rate and the cumulative distribution function is then obtained by integrating the density function for progressively longer times. To obtain the cumulative sequestration-time distribution for the stock of regenerated DIC, we first integrate the tracer field over the whole volume of the ocean and then integrate the resulting stock for progressively longer sequestration times. By year 10,000, the resulting integral is equal to the global inventory of regenerated DIC.

Data availability

Supporting data used to run the inverse model are available at https://doi.org/10.5281/zenodo.10016054. Model output from the inverse model is available at https://doi.org/10.5281/zenodo.8253973. Source data are provided with this paper.

Code availability

The code for the inverse model is available at https://doi.org/10.5281/zenodo.8368856.

References

1. Boyd, P. W., Claustre, H., Levy, M., Siegel, D. A. & Weber, T. Multi-faceted particle pumps drive carbon sequestration in the ocean. Nature 568, 327–335 (2019).

2. Henson, S. A. et al. A reduced estimate of the strength of the ocean’s biological carbon pump. Geophys. Res. Lett. 38, L04606 (2011).

3. Henson, S. A. et al. Uncertain response of ocean biological carbon export in a changing world. Nat. Geosci. 15, 248–254 (2022).

4. Huang, Y., Fassbender, A. J. & Bushinsky, S. M. Biogenic carbon pool production maintains the Southern Ocean carbon sink. Proc. Natl Acad. Sci. USA 120, e2217909120 (2023).

5. Dunne, J. P., Armstrong, R. A., Gnanadesikan, A. & Sarmiento, J. L. Empirical and mechanistic models for the particle export ratio. Global Biogeochem. Cycles 19, GB4026 (2005).

6. Laws, E. A., D’Sa, E. & Naik, P. Simple equations to estimate ratios of new or export production to total production from satellite-derived estimates of sea surface temperature and primary production. Limnol. Oceanogr. Methods 9, 593–601 (2011).

7. Guidi, L. et al. A new look at ocean carbon remineralization for estimating deepwater sequestration. Global Biogeochem. Cycles 29, 1044–1059 (2015).

8. Palevsky, H. I. & Doney, S. C. How choice of depth horizon influences the estimated spatial patterns and global magnitude of ocean carbon export flux. Geophys. Res. Lett. 45, 4171–4179 (2018).

9. Henson, S. A., Sanders, R. & Madsen, E. Global patterns in efficiency of particulate organic carbon export and transfer to the deep ocean. Global Biogeochem. Cycles 26, GB1028 (2012).

10. Maiti, K., Charette, M. A., Buesseler, K. O. & Kahru, M. An inverse relationship between production and export efficiency in the Southern Ocean. Geophys. Res. Lett. 40, 1557–1561 (2013).

11. Cavan, E. L. et al. Attenuation of particulate organic carbon flux in the Scotia Sea, Southern Ocean, is controlled by zooplankton fecal pellets. Geophys. Res. Lett. 42, 821–830 (2015).

12. Laurenceau-Cornec, E. C. et al. The relative importance of phytoplankton aggregates and zooplankton fecal pellets to carbon export: insights from free-drifting sediment trap deployments in naturally iron-fertilised waters near the Kerguelen Plateau. Biogeosciences 12, 1007–1027 (2015).

13. Le Moigne, F. A. C. et al. What causes the inverse relationship between primary production and export efficiency in the Southern Ocean? Geophys. Res. Lett. 43, 4457–4466 (2016).

14. Emerson, S. Annual net community production and the biological carbon flux in the ocean. Global Biogeochem. Cycles 28, 14–28 (2014).

15. Lauvset, S. K. et al. An updated version of the global interior ocean biogeochemical data product, GLODAPv2.2021. Earth Syst. Sci. Data 13, 5565–5589 (2021).

16. Hansell, D. A. et al. Compilation of dissolved organic matter (DOM) data obtained from global ocean observations from 1994 to 2021. Version 2 (NCEI Accession 0227166). NOAA National Centers for Environmental Information, https://doi.org/10.25921/s4f4-ye35 (2021).

17. Dall Olmo, G., Dingle, J., Polimene, L., Brewin, R. J. W. & Claustre, H. Substantial energy input to the mesopelagic ecosystem from the seasonal mixed-layer pump. Nat. Geosci. 9, 820–825 (2016).

18. Omand, M. M. et al. Eddy-driven subduction exports particulate organic carbon from the spring bloom. Science 348, 222–225 (2015).

19. Hansell, D. A., Carlson, C. A., Repeta, D. J. & Schlitzer, R. Dissolved organic matter in the ocean: a controversy stimulates new insights. Oceanography 22, 202–211 (2009).

20. Roshan, S. & DeVries, T. Efficient dissolved organic carbon production and export in the oligotrophic ocean. Nat. Commun. 8, 2036 (2017).

21. Steinberg, D. K. & Landry, M. R. Zooplankton and the ocean carbon cycle. Annu. Rev. Mar. Sci. 9, 413–444 (2017).

22. Jónasdóttir, S. H., Visser, A. W., Richardson, K. & Heath, M. R. Seasonal copepod lipid pump promotes carbon sequestration in the deep North Atlantic. Proc. Natl Acad. Sci. 112, 12122–12126 (2015).

23. DeVries, T. & Weber, T. The export and fate of organic matter in the ocean: new constraints from combining satellite and oceanographic tracer observations. Global Biogeochem. Cycles 31, 535–555 (2017).

24. Nowicki, M., DeVries, T. & Siegel, D. A. Quantifying the carbon Export and Sequestration Pathways of the Ocean’s Biological Carbon Pump. Global Biogeochem. Cycles 36, e2021GB007083 (2022).

25. Bianchi, D., Weber, T. S., Kiko, R. & Deutsch, C. Global niche of marine anaerobic metabolisms expanded by particle microenvironments. Nat. Geosci. 11, 263–268 (2018).

26. Fu, W., Randerson, J. T. & Moore, J. K. Climate change impacts on net primary production (NPP) and export production (EP) regulated by increasing stratification and phytoplankton community structure in the CMIP5 models. Biogeosciences 13, 5151–5170 (2016).

27. Mouw, C. B., Barnett, A., McKinley, G. A., Gloege, L. & Pilcher, D. Global ocean particulate organic carbon flux merged with satellite parameters. Earth Syst. Sci. Data 8, 531–541 (2016).

28. Siegel, D. A., DeVries, T., Cetinic, I. & Bisson, K. M. Quantifying the ocean’s biological pump and Its carbon cycle impacts on global scales. Annu. Rev. Mar. Sci. 15, 329–356 (2023).

29. Le Moigne, F. A. Pathways of organic carbon downward transport by the oceanic biological carbon pump. Front. Mar. Sci. 6, 634 (2019).

30. Cochran, J. K. et al. Time-series measurements of 234Th in water column and sediment trap samples from the northwestern Mediterranean Sea. Deep Sea Res. II Top. Stud. Oceanogr. 56, 1487–1501 (2009).

31. Primeau, F. W., Holzer, M. & DeVries, T. Southern Ocean nutrient trapping and the efficiency of the biological pump. J. Geophys. Res. Oceans 118, 2547–2564 (2013).

32. Hansell, D. A., Carlson, C. A. & Schlitzer, R. Net removal of major marine dissolved organic carbon fractions in the subsurface ocean. Global Biogeochem. Cycles 26, GB1016 (2012).

33. Gardner, W. D., Chung, S. P., Richardson, M. J. & Walsh, I. D. The oceanic mixed-layer pump. Deep Sea Res. II Top. Stud. Oceanogr. 42, 757–775 (1995).

34. Levy, M. et al. Physical pathways for carbon transfers between the surface mixed layer and the ocean interior. Global Biogeochem. Cycles 27, 1001–1012 (2013).

35. Quay, P., Emerson, S. & Palevsky, H. Regional pattern of the ocean’s biological pump based on geochemical observations. Geophys. Res. Lett. 47, e2020GL088098 (2020).

36. Yang, B., Emerson, S. R. & Quay, P. D. The subtropical ocean’s biological carbon pump determined from O_2 and DIC/DI^13C tracers. Geophys. Res. Lett. 46, 5361–5368 (2019).

37. Steinberg, D. K. et al. Bacterial vs. zooplankton control of sinking particle flux in the ocean’s twilight zone. Limnol. Oceanogr. 53, 1327–1338 (2008).

38. Frigstad, H. et al. Links between surface productivity and deep ocean particle flux at the Porcupine Abyssal Plain sustained observatory. Biogeosciences 12, 5885–5897 (2015).

39. Giering, S. L. et al. Reconciliation of the carbon budget in the ocean’s twilight zone. Nature 507, 480–483 (2014).

40. Cram, J. A. et al. The role of particle size, ballast, temperature, and oxygen in the sinking flux to the deep sea. Global Biogeochem. Cycles 32, 858–876 (2018).

41. Weber, T., Cram, J. A., Leung, S. W., DeVries, T. & Deutsch, C. Deep ocean nutrients imply large latitudinal variation in particle transfer efficiency. Proc. Natl Acad. Sci. USA 113, 8606–8611 (2016).

42. Marsay, C. M. et al. Attenuation of sinking particulate organic carbon flux through the mesopelagic ocean. Proc. Natl Acad. Sci. USA 112, 1089–1094 (2015).

43. Kwon, E. Y., Primeau, F. & Sarmiento, J. L. The impact of remineralization depth on the air–sea carbon balance. Nat. Geosci. 2, 630–635 (2009).

44. Crichton, K. A. et al. What the geological past can tell us about the future of the ocean’s twilight zone. Nat. Commun. 14, 2376 (2023).

45. Laws, E. A., Falkowski, P. G. Jr, S, W. O., Ducklow, H. & McCart, J. J. Temperature effects on export production in the open ocean. Global Biogeochem. Cycles 14, 1231–1246 (2000).

46. Siegel, D. A. et al. Global assessment of ocean carbon export by combining satellite observations and food-web models. Global Biogeochem. Cycles 28, 181–196 (2014).

47. Le Moigne, F. A. C. et al. Sequestration efficiency in the iron-limited North Atlantic: implications for iron supply mode to fertilized blooms. Geophys. Res. Lett. 41, 4619–4627 (2014).

48. Manabe, S. & Stouffer, R. J. Century-scale effects of increased atmospheric CO_2 on the ocean–atmosphere system. Nature 364, 215–218 (1993).

49. Liu, Y., Moore, J., Primeau, F. & Wang, W. Reduced CO_2 uptake and growing nutrient sequestration from slowing overturning circulation. Nat. Clim. Change 13, 83–90 (2023).

50. Li, G. et al. Increasing ocean stratification over the past half-century. Nat. Clim. Change 10, 1116–1123 (2020).

51. Letscher, R. T. & Moore, J. K. Preferential remineralization of dissolved organic phosphorus and non-Redfield DOM dynamics in the global ocean: impacts on marine productivity, nitrogen fixation, and carbon export. Global Biogeochem. Cycles 29, 325–340 (2015).

52. Westberry, T., Behrenfeld, M. J., Siegel, D. A. & Boss, E. Carbon-based primary productivity modeling with vertically resolved photoacclimation. Global Biogeochem. Cycles 22, n/a–n/a (2008).

53. Garcia, H. et al. World Ocean Atlas 2018. Vol. 4: dissolved inorganic nutrients (phosphate, nitrate and nitrate+nitrite, silicate). NOAA Atlas NESDIS 84, https://archimer.ifremer.fr/doc/00651/76336/ (2019).

54. Locarnini, M. et al. World Ocean Atlas 2018, volume 1: temperature. NOAA Atlas NESDIS 81, https://archimer.ifremer.fr/doc/00651/76338/ (2018).

55. Moore, J. K. et al. Sustained climate warming drives declining marine biological productivity. Science 359, 1139–1143 (2018).

56. Meinshausen, M. et al. Historical greenhouse gas concentrations for climate modelling (CMIP6). Geosci. Model Dev. 10, 2057–2116 (2017).

57. Keeling, C. D. et al. in A History of Atmospheric CO_2 and its Effects on Plants, Animals, and Ecosystems (eds Baldwin, I. et al.) 83–113 (Springer, 2005).

58. Galbraith, E. D. & Martiny, A. C. A simple nutrient-dependence mechanism for predicting the stoichiometry of marine ecosystems. Proc. Natl Acad. Sci. USA 112, 8199–8204 (2015).

59. DeVries, T. & Holzer, M. Radiocarbon and helium isotope constraints on deep ocean ventilation and mantle‐3He sources. J. Geophys. Res. Oceans 124, 3036–3057 (2019).

60. Kriest, I. & Oschlies, A. On the treatment of particulate organic matter sinking in large-scale models of marine biogeochemical cycles. Biogeosciences 5, 55–72 (2008).

61. Najjar, R. G. et al. Impact of circulation on export production, dissolved organic matter, and dissolved oxygen in the ocean: results from Phase II of the Ocean Carbon-cycle Model Intercomparison Project (OCMIP-2). Global Biogeochem. Cycles 21, GB3007 (2007).

62. Sarmiento, J. L. & Gruber, N. Ocean Biogeochemical Dynamics (Princeton Univ. Press, 2006).

63. Huang, B. et al. NOAA Extended reconstructed sea surface temperature (ERSST), version 5. NOAA National Centers for Environmental Information, https://doi.org/10.7289/V5T72FNM (2017).

64. Graven, H. et al. Compiled records of carbon isotopes in atmospheric CO_2 for historical simulations in CMIP6. Geosci. Model Dev. 10, 4405–4417 (2017).

65. Hua, Q. et al. Atmospheric radiocarbon for the period 1950–2019. Radiocarbon 64, 723–745 (2022).

66. Wang, W. L., Moore, J. K., Martiny, A. C. & Primeau, F. W. Convergent estimates of marine nitrogen fixation. Nature 566, 205–211 (2019).

Acknowledgements

We thank the thousands of scientists and researchers who made hydrographic and organic carbon measurements. We also thank the people and institutes who maintain the public data repositories (GLODAP and WOA) and the Ocean Productivity website maintained by Oregon State University. W.-L.W. was supported by the National Natural Science Foundation of China (42330401) and the Natural Science Foundation of Fujian Province of China 2023J02001. F.W.P. was supported by U.S. National Science Foundation (NSF) OCE-1948842 and OCE-2424014 and U.S. Department of Energy (DOE) DE-SC0021267. W.F. was supported by U.S. DOE DE-SC0021267. Further support for W.F. and Y.L. was provided by the U.S. DOE Reducing Uncertainty in Biogeochemical Interactions through Synthesis and Computation (RUBISCO) Scientific Focus Area (SFA). R.T.L. was supported by U.S. NSF 1829916 and U.S. DOE DE-SC0022177. J.-M.T. was supported by the National Natural Science Foundation of China (92058204 and 41721005) and the Ph.D. Fellowship of the State Key Laboratory of Marine Environmental Science at Xiamen University.

Author information

Author notes

1. Weiwei Fu

Present address: Department of Atmospheric and Oceanic Science, Fudan University, Shanghai, China

2. Yi Liu

Present address: Department of Geosciences, Princeton University, Princeton, NJ, USA

Authors and Affiliations

1. State Key Laboratory of Marine Environmental Science, College of Ocean and Earth Sciences, Xiamen University, Xiamen, China

Wei-Lei Wang & Jin-Ming Tang

2. Department of Earth System Science, University of California, Irvine, Irvine, CA, USA

Weiwei Fu, Yi Liu & François W. Primeau

3. Univ Brest, CNRS, IRD, Ifremer, LEMAR, Plouzané, France

Frédéric A. C. Le Moigne

4. Earth Sciences and Ocean Process Analysis Laboratory, University of New Hampshire, Durham, NH, USA

Robert T. Letscher

Contributions

W.-L.W. conceived the project, with input from F.W.P. W.-L.W. built the inverse model, with contributions from W.F. and F.W.P. W.-L.W. and F.W.P. carried out the formal analyses, with contributions from W.F. W.-L.W. and F.W.P. wrote the manuscript, with input from W.F., F.A.C.L.M., R.T.L., Y.L. and J.-M.T. All authors contributed to the discussion and commented on the manuscript.

Corresponding authors

Correspondence to Wei-Lei Wang or François W. Primeau.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature thanks Stephanie Henson, Shirley Leung and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Sampling density of DIP, DIC, ALK and O[2].

a, DIP. b, DIC. c, ALK. d, O_2. The observational data, downloaded from the GLODAPv2 (ref. [15]), are binned to the OCIM grid. The colour denotes the fraction of the grid boxes in each water column with at least one measurement. For each vertical column, the sampling density is defined as the number of grid boxes with at least one sample divided by the total number of wet grid boxes.

Source Data

Extended Data Fig. 2 Spatial and monthly DOC sampling density.

a, Spatial DOC sampling density. b,c, Monthly DOC sampling density. The DOC observations, obtained from a recent compilation[16], are interpolated to the OCIM grid. The colour denotes the fraction of the grid boxes in each water column with at least one measurement. For each water column, the sampling density is defined as the total number of grid boxes with at least one sample divided by the total number of wet grid boxes.

Source Data

Extended Data Fig. 3 Tracer–tracer comparison for DIP, ALK, DIC and O[2].

a–d, Observations and optimal model. e–h, Model constrained using only O_2 and DOC based on the SeaWiFS CAFE NPP field. The plot shows the joint density distribution for the modelled and observed tracer concentrations. The volume under the distribution integrates to 100th percentiles. The colour indicates the fraction of the distribution that falls outside the given contour. The dashed red line shows the one-to-one line. The optimal model captures 93%, 87%, 94% and 88% of the spatial variance of the GLODAPv2 DIP, ALK, DIC and O_2 data, respectively, whereas the model constrained using only DOC and O_2 captures 66%, 0.0%, 56% and 89% of the spatial variance of the GLODAPv2 DIP, ALK, DIC and O_2 data, respectively.

Source Data

Extended Data Fig. 4 Comparison of model DOC (DOC + DOC[r] + DOC[l]) based on SeaWiFS CbPM NPP field to observations.

a, Tracer–tracer comparison for DOC between the observations and the optimal model. Red circles show DOC observations in the Atlantic Ocean, black squares in the Pacific Ocean, blue triangles in the Indian Ocean and green stars in the Arctic Ocean. The red line shows the one-to-one line. The model captures roughly 80% of the spatial variance of the DOC data. b–d, Comparisons of model DOC to those measured at ocean stations at different depths. The in situ DOC measurements are interpolated to the model grid. The numbers above/below each box represent the number of measurements at each depth. The box plots summarize the distributions of in situ measurements, which show the 25th, 50th and 75th percentiles binned according to the DOC concentration. The whiskers cover 99.3% of the data, with the remaining points shown as red crosses.

Source Data

Extended Data Fig. 5 DIC and DI[14]C observations.

a, Number of hydrographic DIC measurements per month in the GLODAPv2 database as a function of time. b, Number of hydrographic δ^14C measurements per month in the GLODAPv2 database as a function of time. c, Estimated excess DIC (DIC(t) − DIC(1850)) computed by averaging the DIC concentration of our optimized model over the top 100 m (blue) and below 100 m (red). For reference, the estimated average background DIC concentration in 1850 was 2,046.8 mmol m−3 for the top 100 m of the water column and 2,308.4 mmol m−3 for the water column below 100 m, implying a reduction in the vertical DIC gradient of approximately 20% owing to the invasion of anthropogenic CO_2 into the ocean. This reduction masks the true strength of the biological pump, unless it is properly accounted for in the model.

Source Data

Extended Data Fig. 6 Model NPP and organic carbon production.

a,b, The NPP patterns based on SeaWiFS CbPM and CAFE products. c,d, The model production of labile organic carbon. e,f, The model production of semi-labile and POC. The left column (a,c,e) is based on the CbPM NPP product and the right column (b,d,f) is based on the CAFE NPP product.

Source Data

Extended Data Fig. 7 Export fluxes and ratios of advective-diffusive to total flux calculated at the base of the model euphotic zone (73.4 m).

a–c, The advective-diffusive flux of labile organic carbon. d–f, The advective-diffusive flux of semi-labile organic carbon. g–i, The non-advective-diffusive flux. j–l, The flux of TOC. m–o, Ratios of advective-diffusive flux to total flux. The left column shows results based on the CbPM NPP product and an e-folding remineralization time of 24 h for labile DOC, whereas the middle and right columns are based on the CAFE NPP product and e-folding remineralization times of 12 h and 24 h for labile DOC, respectively.

Source Data

Extended Data Fig. 8 Distributions of exponent b values for the non-advective-diffusive carbon flux.

a,b, The optimal b-value distributions based on SeaWiFS CbPM and CAFE products, respectively. c,d, The projected change in the b-value according to temperature prediction by a CESM-BGC model prediction under the RCP8.5 scenario in the year 2099 (ref. [51]). Larger b-values implies that respiration occurs nearer the sea surface.

Source Data

Extended Data Fig. 9 Distributions of standard deviations.

a, Standard deviations of TOC flux. b, Standard deviations of non-advective-diffusive flux. c, Standard deviations of advective-diffusive flux by labile DOC. d, Standard deviations of advective-diffusive flux by semi-labile DOC. e, Standard deviations of the ratio of advective-diffusive flux to TOC flux. f, Standard deviations of DOC residence time in years. These standard deviations are computed from four distinct model configurations, which hinge on two distinct NPP products, namely, CbPM and CAFE, along with two varying e-folding remineralization timescales for labile DOC, specifically, 12 h and 24 h.

Source Data

Extended Data Table 1 Most probable model parameter values with their uncertainties (±1σ)

Supplementary information

Peer Review File

Source data

Source Data Fig. 2

Source Data Fig. 3

Source Data Fig. 4

Source Data Fig. 5

Source Data Extended Data Fig. 1

Source Data Extended Data Fig. 2

Source Data Extended Data Fig. 3

Source Data Extended Data Fig. 4

Source Data Extended Data Fig. 5

Source Data Extended Data Fig. 6

Source Data Extended Data Fig. 7

Source Data Extended Data Fig. 8

Source Data Extended Data Fig. 9

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Wang, WL., Fu, W., Le Moigne, F.A.C. et al. Biological carbon pump estimate based on multidecadal hydrographic data. Nature 624, 579–585 (2023). https://doi.org/10.1038/s41586-023-06772-4

- Received: 28 October 2022

- Accepted: 20 October 2023

- Published: 06 December 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06772-4

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Carbon sequestration by multiple biological pump pathways in a coastal upwelling biome

Article Open access 11 April 2023

Michael R. Stukel, John P. Irving, … Natalia Yingling

Uncertain response of ocean biological carbon export in a changing world

Article 08 April 2022

Stephanie A. Henson, Charlotte Laufkötter, … Emma L. Cavan

Changing carbon-to-nitrogen ratios of organic-matter export under ocean acidification

Article 19 October 2020

Jan Taucher, Tim Boxhammer, … Ulf Riebesell

Oceans can capture more carbon dioxide than previously thought

Nature Research Briefing 18 Dec 2023

Article Open access Published: 29 November 2023

Human mobility networks reveal increased segregation in large cities

Hamed Nilforoshan,

Wenli Looi,

Emma Pierson,

Blanca Villanueva,

Nic Fishman,

Yiling Chen,

John Sholar,

Beth Redbird,

David Grusky &

…

Jure Leskovec

Show authors

Nature volume 624, pages 586–592 (2023)

- 27k Accesses

- 886 Altmetric

- Metrics details

Abstract

A long-standing expectation is that large, dense and cosmopolitan areas support socioeconomic mixing and exposure among diverse individuals1,2,3,4,5,6. Assessing this hypothesis has been difficult because previous measures of socioeconomic mixing have relied on static residential housing data rather than real-life exposures among people at work, in places of leisure and in home neighbourhoods7,8. Here we develop a measure of exposure segregation that captures the socioeconomic diversity of these everyday encounters. Using mobile phone mobility data to represent 1.6 billion real-world exposures among 9.6 million people in the United States, we measure exposure segregation across 382 metropolitan statistical areas (MSAs) and 2,829 counties. We find that exposure segregation is 67% higher in the ten largest MSAs than in small MSAs with fewer than 100,000 residents. This means that, contrary to expectations, residents of large cosmopolitan areas have less exposure to a socioeconomically diverse range of individuals. Second, we find that the increased socioeconomic segregation in large cities arises because they offer a greater choice of differentiated spaces targeted to specific socioeconomic groups. Third, we find that this segregation-increasing effect is countered when a city’s hubs (such as shopping centres) are positioned to bridge diverse neighbourhoods and therefore attract people of all socioeconomic statuses. Our findings challenge a long-standing conjecture in human geography and highlight how urban design can both prevent and facilitate encounters among diverse individuals.

Similar content being viewed by others

Main

In the United States, economic segregation is very high, with income affecting where one lives[9], who one marries[10], and who one meets and befriends[11]. This extreme segregation is costly. It reduces economic mobility12,13,14,15, fosters a wide range of health problems16,17,18 and increases political polarization19,20,21,22. Although there are all manner of reforms designed to reduce economic segregation (such as subsidized housing), it has long been argued that one of the most powerful segregation-reducing dynamics is rising urbanization[23] and the resulting happenstance mixing that it induces1,2,3,4,5,6. This ‘cosmopolitan mixing hypothesis’ anticipates that, in large cities, the combination of increased population diversity, constrained space and accessible public transportation will bring diverse individuals into close physical proximity with one another[2], reducing everyday socioeconomic segregation. The New York City Subway has been lauded, for example, as a mixing bowl in which a diverse set of people cross paths each day[24].

As plausible as the cosmopolitan mixing hypothesis might seem, big cities also provide new opportunities for self-segregation, because they are large enough to enable people to seek out and find others who are similar to themselves[25]. These contrasting hypotheses about the relationship between urbanization and socioeconomic mixing remain untested because it has been difficult to measure real-world exposures that take the form of path crossings and encounters among individuals7,26,27. It becomes possible to measure such exposures when mobile phone geolocation data are analysed at the device level. Although mobile phone data have been used for many research purposes28,29,30,31,32,33,34,35,36,37,38, a nationwide study of socioeconomic mixing and urbanization has not been undertaken because of difficulties in ascertaining individual-level socioeconomic status (SES), determining when dyadic exposures occur, and amassing the data needed to compare across cities or counties28,29,30,32,33,34,35,36.

Here we carefully test the cosmopolitan mixing hypothesis and the dynamics underlying it. To assess this hypothesis and understand the relationship between urbanization and segregation, we use mobile phone mobility data in the form of de-identified GPS location pings (see the ‘SafeGraph’ section of the Methods). From this data, we capture geolocated individual-level exposures between individuals of similar or different SES. This enables us to develop city-level and county-level measures of segregation that capture where people go, when they go there and whom they encounter on the way.

We first determine the SES of a person by identifying their home location and its monthly rent value. We next construct a dynamic network that captures each individual’s exposures to other individuals in their everyday life. Our network contains 1,570,782,460 edges (representing exposures in physical space) among 9,567,559 nodes (representing individuals, that is, mobile phones) across 382 MSAs and 2,829 counties in the United States. Every timestamped edge between a pair of nodes signifies that the two individuals crossed paths with and encountered each other (that is, they were at the same location at the same time). We analysed these data to estimate the amount of exposure segregation, defined as the extent to which individuals of different economic statuses are exposed to one another within each geographical area (MSAs and counties) in the United States. Our measure of exposure segregation extends a traditional static segregation measure by capturing the diversity of person-to-person exposures localized in space and time.

A more realistic measure of segregation

To estimate each person’s SES, we first infer their home location from night-time mobile phone location pings (Fig. 1a; see the ‘Inferring home location’ section of the Methods), and we then recover the estimated monthly rent value of the home at this location (Fig. 1a; see the ‘Inferring SES’ section of the Methods). This method is more accurate in estimating individual SES than the conventional approach of using neighbourhood-level census averages30,31. We next identify each instance when a pair of individuals crossed paths and were thus exposed to each other, defined as their two devices being within D metres of each other within T minutes (see the ‘Constructing exposure network’ section of the Methods). Although our key findings are robust to the precise choice of D and T (Supplementary Figs. 5–8), our primary analyses use D = 50 metres and T = 5 minutes because the cosmopolitan mixing hypothesis pertains to visual exposure1,2. This approach, to our knowledge, provides the highest-resolution measure of exposure to date, compared with previous GPS-based studies30,31,39.

Fig. 1: Exposure segregation captures the likelihood of exposure between people of different socioeconomic backgrounds and reveals increased segregation in highly populated metropolitan areas.

a, For 9.6 million individuals (mobile phones), we infer their SES (rent or rent equivalent) from their home address on the basis of their location at night (see the ‘Inferring home location’ section of the Methods). We then capture path-crossing events (that is, being at the same location at the same time) to identify pairs of individuals who were exposed to each other (see the ‘Constructing exposure network’ section of the Methods). b, The nationwide network of 1.6 billion exposures spans 2,829 counties and 382 MSAs. Our exposure network contrasts with a conventional measure of economic segregation, the neighbourhood sorting index, which assumes that individuals are exposed to other residents only within their home census tract. Graphs pertain to a sample community of 50 individuals residing in ten census tracts in San Francisco, CA. Nodes represent individuals; edges represent exposures. This sample illustrates the importance of capturing cross-tract exposures, which are undetected by conventional segregation measures. c, For each geographical region (either MSA or county), we estimate exposure segregation, defined as the correlation between an individual’s SES and the mean SES of those with whom they cross paths; 1 signifies perfect segregation and 0 signifies no segregation. This definition is equivalent to the conventional neighbourhood sorting index, but with the key difference that it leverages real-life exposure from mobility data instead of synthetic exposures from individuals grouped by census tracts. For two MSAs, we show the raw data; each point represents one individual. San Francisco–Oakland–Hayward, CA, is 2.2× more segregated (P < 10−4, 95% CI = 1.6–2.8×; two-sided bootstrap; see the ‘Hypothesis testing’ section of the Methods) than Napa, CA. d,e, Contrary to the hypothesis that highly populated metropolitan areas support diverse exposures and socioeconomic mixing, we find that larger MSAs are more segregated (d). Exposure segregation presented as a function of population size; each dot represents one MSA; the purple line indicates the LOWESS fit. An upward slope reveals that urbanization is associated with higher exposure segregation (Spearman correlation = 0.62, n = 382, P < 10−4; two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). The top ten largest MSAs by population size are 67% more segregated (P < 10−4, 95% CI = 49–87%; two-sided bootstrap; see the ‘Hypothesis testing’ section of the Methods) than small MSAs with fewer than 100,000 residents. Associations are robust to controlling for potential confounding factors and are similar for population density and exposure segregation (Extended Data Table 1 and Supplementary Table 7). e, Exposure segregation across the 2,829 US counties. The analysis was limited to counties with at least 50 individuals present in the dataset. Exposure segregation varies substantially across counties in the United States. Moreover, as with MSA-level segregation, county-level exposure segregation is also positively associated with both population size and population density (Extended Data Fig. 4).

The economic segregation of each geographical region is measured by the correlation between a person’s SES and the mean SES of everyone to whom they are exposed through a path crossing (see the ‘Exposure segregation’ section of the Methods). This correlation is estimated by fitting a linear mixed-effects model that eliminates attenuation bias and secures unbiased estimates of exposure segregation even when observed exposures are sparse (Extended Data Fig.  1; see the ‘Estimating exposure segregation’ section of the Methods). The resulting measure of exposure segregation (Fig. 1b,c), which ranges from 0 (perfect integration) to 1 (complete segregation), is a generalization of a widely used measure of socioeconomic segregation—the neighbourhood sorting index[7]. The neighbourhood sorting index is equivalent to the correlation between each person’s SES and the mean SES of all of the people in their home census tract, whereas our measure of exposure segregation is equivalent to the correlation between each person’s SES and the mean SES of all of the people who they encounter (either inside or outside their home census tract). Thus, the key difference between these two measures is that the neighbourhood sorting index assumes that exposures occur uniformly and only among co-residents of the same home tract, whereas exposure segregation captures real-world exposures among people as they navigate their daily lives.

Extreme segregation in large cities

We find that, contrary to the cosmopolitan mixing hypothesis, exposure segregation is higher in large MSAs (Fig. 1d). The Spearman correlation between MSA population and MSA segregation is 0.62 (P < 10−4), and the ten largest MSAs by population size are 67% more segregated (P < 10−4, 95% confidence interval (CI) = 49–87%) than small MSAs with fewer than 100,000 residents. This result is robust. We validated it by recalculating the correlation with a measure of density rather than population size (Spearman correlation = 0.45, P < 10−4; Supplementary Table 7), by controlling for potential confounding factors (Extended Data Table 1 and Supplementary Table 7), by varying the granularity of the analysis (Fig. 1e and Extended Data Fig. 4) and by testing a variety of specifications of exposure segregation (Supplementary Table 6 and Supplementary Figs. 2–10). The consistent result that larger, denser cities are more segregated runs counter to the hypothesis that such cities promote socioeconomic mixing by attracting diverse individuals and constraining space in ways that oblige them to encounter one other1,2,3,4,5,6. Our results support the opposite hypothesis: big cities allow their inhabitants to seek out people who are more like themselves. The key advance that enables this finding is our fine-grained measure of proximity with respect to both time and space (Supplementary Fig. 66).

Exploring exposure segregation

Our methodology further allows for comparisons between a conventional static measure of segregation (neighbourhood sorting index) and our dynamic measure. The median level of exposure segregation across all MSAs is 38% lower (P < 10−4, 95% CI = 37–41%) than the corresponding value for a conventional static estimate[31] (neighbourhood sorting index; Fig. 2a (top)). We explain this result by disaggregating our measure into components pertaining to exposures in which both, one or neither individual was within their home census tract (Fig. 2a (bottom)). Exposure segregation is lower because, when people venture outside their home tracts, they experience more diversity. For example, exposures are 50% less segregated (P < 10−4, 95% CI 48–53%) when both people are outside the home census tract than when both people are within their home tract. Within their own neighbourhood, people cross paths with neighbours who are socioeconomically most similar to them, but this has little effect on overall exposure segregation because only 2.4% of exposures (95% CI = 2.4–2.4%) occur when both individuals are within their home tract. Finally, we observe that not only is overall exposure segregation elevated in large cities, but also each of its components is elevated in large cities (Supplementary Fig. 10).

Fig. 2: Exploring the dynamics of exposure segregation reveals that socioeconomic differentiation of spaces accounts for increased segregation in large cities.

a, Each point represents the segregation estimate in one of the n = 382 MSAs; the vertical coloured lines represent the median across MSAs. Top, exposure segregation is 38% lower (P < 10−4, 95% CI = 37–41%; two-sided bootstrap; see the ‘Hypothesis testing’ section of the Methods) than the conventional segregation measure—the neighbourhood sorting index. Bottom, a breakdown of exposure segregation into its component parts. Exposures in which both people are within their home census tract (green) are most segregated, reflecting the homophily effect in which people preferentially encounter those of a similar SES in their home tracts. Out-of-tract exposures (orange and red) are less segregated, reflecting the visitor effect in which entering other tracts exposes individuals to economically diverse individuals. As a small minority (2.4%, 95% CI = 2.4–2.4%; two-sided bootstrap; see the ‘Hypothesis testing’ section of the Methods) of exposures happen within the home tract, the visitor effect dominates the homophily effect and exposure segregation is therefore lower than the conventional neighbourhood sorting index. b,c, Exposure segregation varies by tie strength and location type. Each point represents segregation in one of n = 382 MSAs using only exposure pairs occurring with a specific tie strength (b) or in a given location type (c). The boxes indicate the interquartile range across MSAs. Segregation increases with tie strength and is especially high for the strongest ties (5+ exposures; median exposure segregation, 0.57). Segregation is highest at golf courses and country clubs (median exposure segregation, 0.42) and lowest at performing arts centres (median exposure segregation, 0.16) and stadiums (median exposure segregation, 0.17). d–f, A case study of full-service restaurants illustrates the relationship between urbanization and exposure segregation. Highly populated metropolitan areas are more segregated not only because they offer a wider choice of venues but also because these venues are more socioeconomically differentiated. d, Larger MSAs have more restaurants within 10 km of the average resident, giving residents more options to self-segregate. e, Moreover, restaurants in larger MSAs vary more in the median SES of their visitors, meaning that a greater choice of socioeconomically differentiated restaurants is offered. The coefficient of variation across restaurant SES (that is, the median SES of a restaurant’s visitors) in the ten largest MSAs is 63% more (P < 10−4, 95% CI = 37–100%; two-sided bootstrap; see the ‘Hypothesis testing’ section of the Methods) than the coefficient of variation in small MSAs (with fewer than 100,000 residents). f, Consequently, exposure segregation within restaurants is higher in larger MSAs. These relationships are also detectable at the scale of city hubs (defined as higher-level clusters of POIs such as plazas and shopping centres) as well as at the neighbourhood level (Extended Data Figs. 5 and 6).

We quantify variability in exposure segregation both by tie strength (Fig. 2b,c) and across different points of interest (POIs). Stronger ties are more segregated40,41 (Fig. 2b). We also find much variability in POI-level segregation11,30 (Fig. 2c; see the ‘Decomposing segregation by activity’ section of the Methods). We explain this variability in POI-level segregation (Fig. 2c) by the extent to which a POI category (such as restaurants) contains differentiated POIs that service small and thereby socioeconomically homogeneous communities (for example, Michelin star restaurants). We operationalize the extent of a POI category’s differentiation using the average travel distance to the nearest POI[30] and the total number of POIs (Spearman correlation = −0.75, P < 0.001 (travel distance); Spearman correlation = 0.69, P < 0.01 (number of POIs); Extended Data Fig. 3a,b). For example, in the median MSA, religious organizations require 92% less travel distance (P < 10−4, 95% CI = 92–93%) and are 16× more numerous (P < 10−4, 95% CI = 8–18×) than stadiums. Because religious organizations can therefore target more narrowly defined socioeconomic communities, they are 75% more segregated (P < 10−4, 95% CI = 58–87%) than stadiums. In rare cases, a POI category with only a small number of POIs may still exhibit substantial segregation (such as golf courses) owing to economic differentiation among its POIs caused by other factors (such as a public–private distinction; Extended Data Fig. 3c). Below, we show that this link between the socioeconomic differentiation of spaces and segregation is also critical to explaining why large cities are more segregated.

Differentiation of space in large cities

To understand why large metropolitan areas support segregation, we present an example of segregation within leisure POIs. Full-service restaurants provide an illustrative example (Fig. 2d–f) of a segregation-inducing dynamic that holds widely across other leisure sites (Supplementary Fig. 22) and other scales of analysis (Extended Data Figs. 5 and 6). We find that larger MSAs offer their residents a greater number of leisure choices: the average resident of one of the ten largest MSAs has 22× more restaurants (P < 10−4, 95% CI = 11–39×) within 10 km of their home compared with an average resident of a small MSA (where a ‘small MSA’ is defined as one with fewer than 100,000 residents; Fig. 2d). These choices are also more socioeconomically differentiated. When a restaurant’s SES is defined as the median SES of all people who visited it and encountered another person, the coefficient of variation of ‘restaurant SES’ in the ten largest MSAs is 63% greater (P < 10−4, 95% CI = 37–100%) than that in small ones (Fig. 2e). Thus, not only do large MSAs offer their residents a larger choice of restaurants, but these restaurants are also more socioeconomically differentiated. For example, in large cities such as New York, one can spend US$10, US$100 or US$1,000 on a meal, depending on the choice of restaurant42,43. These processes mean that exposure segregation in restaurants is 29% higher (P < 10−3, 95% CI 8–49%) in the ten largest MSAs than in small MSAs (Fig. 2f). We find analogous results across many POI types (Supplementary Fig. 22) and at higher levels of scale pertaining to city hubs (for example, plazas, shopping centres, boardwalks) as well as neighbourhoods (Extended Data Figs. 5 and 6).

Mitigating segregation through urban design

Our results suggest that segregation could be mitigated when frequently visited POIs, which we refer to as ‘hubs’, are positioned in close proximity to diverse neighbourhoods. These hubs would serve as bridges between residents of nearby high-SES and low-SES neighbourhoods, enabling them to easily visit the hubs44,45,46 and encounter one another (Fig. 3c). We developed the bridging index (see the ‘Bridging index’ section of the Methods) to measure whether hubs are located in such bridging positions. Our index measures the economic diversity of the groups that would encounter each other if everybody visited only their nearest hub. It is computed by clustering individuals by the nearest hub to their home and then measuring the economic diversity within these clusters (Extended Data Fig. 7). The resulting index ranges from 0 to 1, where 0 means that individuals near each hub have a uniform SES, and 1 means that individuals near each hub are as diverse as the overall area (Extended Data Fig. 8). We compute our bridging index for commercial centres (such as plazas, shopping centres, boardwalks) because we find that they are common hubs of exposure: the majority (56.9%, 95% CI = 56.9–56.9%) of exposures across all 382 MSAs occur in close proximity (within 1 km) to a commercial centre, even though only 2.5% of land area is within 1 km of a commercial centre (Fig. 3c). The results show that our bridging index is strongly associated with exposure segregation (Spearman correlation = −0.78, P < 10−4; Fig. 3d). The top ten MSAs with the highest bridging index are 53.1% less segregated (P < 10−4, 95% CI = 44–60%) than the ten MSAs with the lowest bridging index. This finding is again robust: the hub-bridging effect is strong and significant (P < 10−4) even after including controls for race, population size, economic inequality and many other variables (Extended Data Tables 2 and 3, Supplementary Table 6 and Supplementary Figs. 2, 8 and  13). It follows that zoning laws and related policies that encourage developers to locate hubs, such as shopping centres, between diverse residential neighbourhoods may reduce exposure segregation. We have identified several large cities that increase integration in this manner (Supplementary Table 21) and present an illustrative example (Fig. 3c,d) in which well-placed hubs bridge diverse individuals in Fayetteville, North Carolina.

Fig. 3: Exposure segregation is lower when frequently visited hubs bridge socioeconomically diverse neighbourhoods.

a, We developed an index (see the ‘Bridging index’ section of the Methods) to quantify the extent to which highly visited hubs bridge socioeconomically diverse neighbourhoods. The metric was constructed by clustering homes by the nearest hub, then measuring the within-cluster diversity of SES. Two plots illustrate that the bridging index is distinct from conventional measures of residential segregation such as the neighbourhood sorting index. The bridging index ranges from 0 (no bridging; top) to 1 (perfect bridging; bottom), while residential segregation is constant (high-SES and low-SES individuals are highly segregated by census tract, denoted by purple and yellow bounding boxes). We compute our bridging index with hubs defined as commercial centres (such as shopping centres and plazas) because the majority (56.9%, 95% CI = 56.9–56.9%; bootstrapping; see the ‘Hypothesis testing’ section of the Methods) of exposures across all 382 MSAs occur in close proximity (within 1 km) to a commercial centre, even though only 2.5% of land area is within 1 km of a commercial centre. b, Our bridging index strongly predicts exposure segregation (Spearman correlation = −0.78, n = 382, P < 10−4; two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). The top ten MSAs with the highest bridging index are 53.1% less segregated (P < 10−4, 95% CI = 44–60%; two-sided bootstrap; see the ‘Hypothesis testing’ section of the Methods) than the ten MSAs with the lowest bridging index. The bridging index predicts segregation more accurately (P < 10−4; two-sided Steiger’s Z-test; see the ‘Hypothesis testing’ section of the Methods) than population size, SES inequality, neighbourhood sorting index and race, and is significantly associated (P < 10−4; two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods) with exposure segregation after controlling for these variables and other potential confounding factors (Extended Data Tables 2 and 3). c,d, A case study of Fayetteville, North Carolina, an MSA with low exposure segregation (21st percentile) despite having an above-median population size (64th percentile) and income inequality (60th percentile). c, Exposure heat map of Fayetteville; all visually discernible hubs are associated with one or more commercial centres. d, Hubs are located in accessible proximity to both high-SES and low-SES census tracts (bridging index = 0.90, 62nd percentile), leading to diverse exposures. An illustrative example of one hub (Highland Center) in Fayetteville and a random sample of ten exposures occurring inside of it. The home icons demarcate home locations of individuals (up to 100 m of random noise was added for anonymity); the colours denote individual and mean tract SES. The maps in c and d were generated using OpenStreetMap data.

Discussion

As big cities continue to grow and spread, it is important to examine whether they encourage socioeconomic mixing. Although it is often argued that big cities promote mixing by increasing density, we find that exposure diversity and city size are negatively related. This result means that scale matters. We have shown that, because large cities can sustain venues that are targeted to thin socioeconomic slices of the population, they have become homophily-generating machines that are far more segregated than small cities. We also find that some cities are able to mitigate this segregative effect because their hubs are located in bridging zones that can draw in people from diverse neighbourhoods. We were able to detect these pockets of homophily (and the counteracting effects of bridging hubs) because we have developed a dynamic measure of economic segregation that captures everyday socioeconomic mixing at home, work and leisure.

This new methodology for measuring exposure segregation, while an improvement over static approaches, has limitations. For example, it is difficult to ascertain how weak or strong the ties are, as we are obliged to use physical proximity as a proxy for exposure[47]. It is reassuring in this regard that our core results persist under stricter time, distance and tie-strength thresholds (Supplementary Table 6 and Supplementary Figs. 5–8), and are associated with key downstream outcomes (Extended Data Fig. 2 and Supplementary Fig. 24). It is likewise important to locate and analyse supplementary datasets that cover subpopulations (for example, subpopulations of homeless individuals) that are not as well represented in our dataset[48]. The available evidence indicates that our sample is well balanced on many key racial, economic and demographic variables[49], but mobile phone market penetration is still not complete, and GPS ping data are unevenly distributed by time. Finally, our measure of SES relies on housing consumption, an indicator that does not exhaust the concept of SES. It is again reassuring that our analytical approach, which improves on conventional neighbourhood-level imputations, is robust under a range of alternative measures of SES (Supplementary Fig. 3).

This is all to suggest that dynamic segregation data are rich enough to overcome many seeming limitations. The dynamic approach that we have taken here could further be extended to examine cross-population differences in the sources of segregation and to develop a more complete toolkit of approaches for reducing segregation and improving urban design.

Methods

The Methods is structured as follows. In the ‘Datasets’ section, we explain the datasets used in our analysis; in the ‘Data processing’ section, we explain the data processing procedures that we use to infer SES and exposures; and in the ‘Analysis’ section, we explain the analyses underlying our main results.

Datasets

SafeGraph

Our primary mobility and location data comprise GPS locations from a sample of adult smartphone users in the United States, provided by the company SafeGraph. The data are de-identified GPS location pings from smartphone applications that are collected and transmitted to SafeGraph by participating users[50]. As described by SafeGraph in the public documentation, SafeGraph data are collected by “partner[ing] with mobile applications that obtain opt-in consent from users to collect anonymous location data. This data is not associated with any name or email address”. SafeGraph ensures that its mobile application partners obtain consent for data to be used for commercial and research purposes, including academic publication. SafeGraph users are able to opt out of data collection at any time.

Although the sample is not random, previous work has demonstrated that SafeGraph data are geographically well balanced (that is, an approximately unbiased sample of different census tracts within each state) and well balanced along the dimensions of race, income and education49,51. Furthermore, SafeGraph data are a widely used standard in large-scale studies of human mobility across many different areas including COVID-19 modelling[51], political polarization[39] and consumer preference tracking[52]. All data provided by SafeGraph were stored on a secure server behind a firewall. Data handling and analysis was conducted in accordance with SafeGraph policies and in accordance with the guidelines of the Stanford University Institutional Review Board.

The raw data consist of 91,755,502 users and 61,730,645,084 pings from three evenly spaced months in 2017: March, July and November. Each ping consists of a latitude, longitude, timestamp, and de-identified user ID. The mean number of raw pings associated with a user is 667 and the median number of pings is 12. We applied several filters to improve the reliability of the SafeGraph data, and subsequently linked each user to an estimated rent (that is, Zillow Zestimate) using their inferred home location (that is, CoreLogic address), as described in the ‘Inferring home location’ and ‘Inferring SES’ sections.

We applied several filters to improve the reliability of the SafeGraph data. To ensure that the locations are reliable, we excluded pings with location estimates less accurate than 100 m, as recommended by SafeGraph[53]. We filtered out users with fewer than 500 pings, as these are largely noise. We also filtered out users for whom we were unable to infer a home, because we rely on home rent values to measure SES. Finally, to avoid duplicate users, we removed users if more than 80% of their pings had identical latitudes, longitudes and timestamps to those of another user; this could potentially occur if, for example, a single person in the real world carries multiple mobile devices. After these initial filters, we were able to infer home locations for 12,183,523 users in the United States (50 states and Washington DC), leveraging the CoreLogic database. Of users for whom we could infer a home location, we were able to successfully link 9,576,650 to an estimated rent value through the Zillow API. The ‘Inferring home location’ and ‘Inferring SES’ sections provide full details on the use of CoreLogic database to infer home locations and the use of the Zillow API to link these home locations to estimated rent values. Finally, after removing users for whom >80% of their pings were duplicates with another user, we reduced the number of users from 9,576,650 to 9,567,559 (that is, we removed less than 0.1% of users through de-duplication).

CoreLogic

We use the CoreLogic real estate database to link users to home locations[54]. The database provides information covering over 99% of US residential properties (145 million properties), over 99% of commercial real estate properties (26 million properties) and 100% of US county, municipal and special tax districts (3,141 counties). The CoreLogic real estate database includes the latitude and longitude of each home, in addition to its full address: street name, number, county, state and zip code.

Zillow

We used the Zillow property database to query for rent estimates[55] (our primary measure of SES). The Zillow database contains rent data (rent Zestimate) for 119 million US residential properties. We were able to determine a rent Zestimate, the primary measure of SES used in our analysis, for 9,576,650 out of 12,183,523 inferred SafeGraph user homes (a 79% hit rate).

SafeGraph Places

Our database of US business establishment boundaries and annotations comes from the SafeGraph Places database[50], which indexes the names, addresses, categories, latitudes, longitudes and geographical boundary polygons of 5.5 million US POIs in the United States. SafeGraph includes the North American Industry Classification System (NAICS) category of each POI, which is standard taxonomy used by the Federal government to classify business establishments[56]. For example, the NAICS code 722511 indicates full-service restaurants. We identified relevant leisure sites using the prefix 7, which includes arts, entertainment, recreation, accommodation and food services, and supplemented these POIs with the prefix 8131 to include religious organizations such as churches. We restricted our analysis of leisure sites to the top-most frequently visited POI categories within these NAICS code prefixes (Fig. 2c): full-service restaurants, snack bars, limited-service restaurants, stadiums and so on. SafeGraph Places also includes higher-level ‘parent’ POI polygons that encapsulate smaller POIs. Specifically, we identified hubs with the NAICS code 531120 (lessors of non-residential real estate), which we find in practice corresponds to commercial centres such as shopping centres, plazas, boardwalks and other clusters of businesses. We provide illustrative examples of such hubs in Supplementary Figs. 16–18.

US census

We extracted demographic and geographical features from the five-year 2013–2017 American Community Survey[57]. This enables us, as described below, to link mobile phone locations to geographical areas including census block group (CBG), census tract and MSA, as well as to infer demographic features corresponding to those demographic areas including median household income.

A CBG is a statistical division of a census tract. CBGs are generally defined to contain between 600 and 3,000 people. A CBG can be identified at the national level by the unique combination of state, county, tract and block group codes.

A census tract is a statistical subdivision of a county containing an average of around 4,000 inhabitants. Census tracts range in population from 1,200 to 8,000 inhabitants. Each tract is identified by a unique numeric code within a county. A tract can be identified at the national level by the unique combination of state, county and tract codes.

Census tracts and block groups typically cover a contiguous geographical area, although this is not a constraint on the shape of the tract or block group. Census tract and block group boundaries generally persist over time so that temporal and geographical analysis is possible across multiple censuses.

Most census tracts and CBGs are delineated by inhabitants who participate in the Census Bureau’s Participant Statistical Areas Program. The Census Bureau determines the boundaries of the remaining tracts and block groups when delineation by inhabitants, local governments or regional organizations is not possible[58].

An MSA is a US geographical area defined by the Office of Management and Budget (OMB) and is one of two types of Core Based Statistical Area (CBSA). A CBSA comprises a county or counties associated with a core urbanized area with a population of at least 10,000 inhabitants and adjacent counties with a high degree of social and economic integration with the core area. Social and economic integration is measured through commuting ties between the adjacent counties and the core. A micropolitan statistical area is a CBSA of which the core has a population of between 10,000 and 50,000; an MSA is a CBSA of which the core has a population of over 50,000. In our primary analysis, we follow a previous study[31] and focus on MSAs, excluding micropolitan statistical areas owing to data sparsity concerns.

TIGER

Road and transportation feature annotations come from the census-curated Topologically Integrated Geographic Encoding and Referencing system (TIGER) database[59]. The TIGER databases are an extract of selected geographical and cartographic information from the US Census Bureau’s Master Address File/Topologically Integrated Geographic Encoding and Referencing (MAF/TIGER) Database (MTDB). We used the MAF/TIGER Feature Class Code (MTFCC) from the TIGER Roads and TIGER Rails databases to identify road and railways. TIGER data are in the format of Shapefiles, which provide the exact boundaries of roads and railways as latitude/longitude coordinates.

Data processing

For each individual, we first infer their home location and subsequently estimate their SES on the basis of their home rent value (see the ‘Inferring home location’ and subsequently ‘Inferring SES’ sections). We then calculate all exposures between individuals (see the ‘Constructing exposure network’ section). We then annotate exposures according to the location in which they occurred. Specifically, we annotate whether the exposure took place in both individuals’ home tract, in one individual’s home tract, or in neither home tract. We also determine whether it occurred inside a fine-grained POI, such as a specific restaurant, as well as whether it took place within a parent POI, like a hub (see the ‘Annotating exposures’ section). Details on all inferences and exposure calculations are provided below.

Inferring home location

We first infer a user’s home latitude and longitude using the latitude and longitude coordinates of their pings during local night-time (and early-morning) hours, based on best practices established by SafeGraph[60]. We first remove users with fewer than 500 pings to ensure that we have enough data to reliably infer home locations. We then interpolate each person’s location for each 1 h window (for example, 18:00–19:00, 19:00–20:00 and 20:00–21:00) using linear interpolation of latitudes and longitudes to ensure that we have time series at a constant time resolution. We perform interpolation using the interpolate package of the scipy library. We filter for hours between 18:00 and 09:00 during which the person moves less than 50 m until the next hour; these stationary night-time (and early-morning) pings represent cases in which the person is more likely to be at home. We filter for users who have such stationary pings on at least three dates and with at least 60% of pings within a 50 m radius. Finally, we infer home latitude and longitude as the median latitude and longitude of these stationary pings (after removing outliers outside the 50 m radius). We choose the thresholds above because they yield a good compromise between inferring the home location of most users and inferring home locations with high confidence. Overall, we are able to infer home locations for 70% of users with more than 500 pings, and these locations are inferred with high confidence; 89% of stationary night-time observations are within 50 m of the inferred home latitude and longitude. Our key findings are robust to the exact choice of threshold for home identification (Supplementary Fig. 62).

Inferring SES

Having inferred each user’s  home location, we link their latitude and longitude to a large-scale housing database (Zillow) to infer the estimated rent of each individual’s home, which we use as a measure of SES. We do this in two steps. First, we link the inferred user’s home latitude and longitude to the CoreLogic property database (see the ‘CoreLogic’ section), a comprehensive database of properties in the United States, by taking the closest CoreLogic residential property (single family residence, condominium, duplex or apartment) to the user’s inferred home latitude and longitude. Second, we use the CoreLogic address to query the Zillow database (see the ‘Zillow’ section), which provides an estimated home rent and price for each individual (the Zillow database does not allow for queries using raw latitude and longitude, which is why it is necessary to leverage to CoreLogic to obtain an address for each user). We use Zillow’s estimated rent for the user’s home as our main measure of SES. We apply several quality-control filters to ensure that the final set of individuals that we use in our main analyses have reliably inferred home locations and SES: (1) we remove a small number of users whose  median latitude and longitude at home are  identical to another user’s, as we empirically observe that these people have unusual ping patterns; (2) we remove users for whom we are lacking a Zillow rent estimate, as this constitutes our primary SES measure; (3) we winsorize Zillow rent estimates that are greater than US$20,000 to avoid spurious results from a small number of outliers; (4) we remove a small number of users who are missing census demographic information for their inferred home location; (5) we remove users whose Zillow home location is further than 100 m from their CoreLogic home location, or whose CoreLogic home location is further than 100 m from their median latitude and longitude at home; (6) we remove a small number of users in single family residences who are mapped to the exact same single family residence as more than 10 other people, as this may indicate a data error in the Zillow database.

The set of users who pass these filters constitutes our final analysis set of 9,567,559 users. We confirm that the census demographic statistics of these users’ inferred home locations are similar to those of the US population in terms of income, age, sex and race.

Any individual quantitative measure provides only a partial picture of a person’s SES. Recognizing this, we conduct robustness checks in which, rather than using the Zillow estimated rent of the user’s home as a proxy for SES, we use (1) the median CBG household income in that area; and (2) the percentile-scored rent of the home, to account for long-tailed rent distributions. Our main results are robust to using these alternative measures of SES (Supplementary Fig. 3).

Constructing the exposure network

We constructed a fine-grained, dynamic exposure network \({\mathcal{G}}\) between all 9,567,559 individuals across 382 MSAs and 2,829 counties, which is represented as an undirected graph \({\mathcal{G}}=({\mathcal{V}},{\mathcal{E}})\) with time-varying edges. Each node \({v}_{i}\in {\mathcal{V}}\) in the graph represents one of the n = 9,567,559 individuals in our study, such that the set of nodes is V = {v_1, v_2, ..., vn}. Each node vi has a single attribute xi, representing the inferred SES (estimated rent) of the individual.

Individuals vi and vj are connected by one edge \({e}_{i,j,k}\in {\mathcal{E}}\) per exposure, with k indicating the kth exposure between individuals vi and vj. Each edge ei,j,k has three attributes ti,j,k, lati,j,k and loni,j,k, indicating the timestamp, latitude and longitude of the exposure, respectively. We now focus our discussion on explaining how each of the exposure edges of the network is calculated.

We define an exposure to occur when two users have GPS pings that are close (according to a fixed threshold) in both physical proximity and time. Specifically if user vi has a GPS ping with ti, lati, loni (indicating the timestamp, latitude and longitude of the ping respectively), and user vj has a GPS ping with tj, latj, lonj, then the users are said to have crossed paths if ∣ti − tj∣ < T and distance((lati, lati), (latj, latj)) < D, where T represents the time threshold (that is, the maximum time distance the two pings can be apart to count as an exposure) and D represents the distance threshold (that is, the maximum physical distance that the two pings can be apart to count as an exposure). We filter for both distance and time simultaneously to ensure that our exposure network includes only pairs of users who are likely to have crossed paths with each other. This high-resolution definition of exposure contrasts with other methods that consider all individuals that visit the same location, irrespective of time[30], to have an equal likelihood of exposure, an unrealistic assumption because the SES of visitors to a given location can vary significantly by time (Supplementary Fig. 63)[61]. This fine-grained measure of proximity with respect to both time and space is the key advance that enables our findings (Supplementary Fig. 66). We use a threshold T of 5 min, which is a stringent threshold on time as the mean number of pings per person per hour during day time is approximately one ping. We use a distance threshold D of 50 m, because the cosmopolitan mixing hypothesis pertains to visual exposure1,2,3 and following previous work showing that even exposure to individuals from afar is linked to long-term outcomes[19]. Our network is validated by correlation to external, gold-standard datasets (Extended Data Fig. 2). Furthermore, we show through a series of robustness checks that our key results in Figs. 1–3 are highly robust to varying thresholds (that is, 1 min or 2 min time thresholds, as well as 10 m or 25 m distance thresholds), as well as additional criteria to increase the tie strength (that is, requiring prolonged exposures, or multiple exposures on unique days). Under all of these different definitions of exposure, our main findings remain consistent (Supplementary Table 6 and Supplementary Figs. 2–8).

To efficiently calculate exposures that occurred among all users, we implement our exposure threshold as a k-dimensional (k-d) tree[62], a data structure that enables one to efficiently identify all pairs of points within a given distance of each other in a k-dimensional space. In total, we identify 1,570,782,460 exposures. The timestamp ti,j,k of the exposure is the minimum ping timestamp in the pair of individuals’ ping timestamps (ti,tj). The location lati,j,k,loni,j,k of the exposure is the average latitude and longitude of pair of pings belonging to the two individuals (lati,latj) and (loni,lonj). We implement our exposure detection system to parallelize across multiple cores, enabling us to efficiently construct the network using a single supercomputer (with 12 TB RAM and 288 cores) in under a week. By contrast, a naive implementation (without k-d trees or parallelization) would necessitate on the order of ~10 years of computing time. The key challenge is accounting for proximity in time and space simultaneously, which results in an O(n^2) time complexity for a naive implementation (where n is the number of pings in the dataset), in contrast to previous work that is time agnostic and can therefore compute exposures using geohashes in O(n) time31,38.

Annotating exposures

Exposures are annotated to indicate whether they occurred at or near POIs, for example, at a user’s home, or within a restaurant. Annotations are not mutually exclusive in that an exposure may be simultaneously tagged as having occurred near multiple POIs from multiple data sources. We describe the specific annotations below.

We annotate a user’s exposure as having occurred at their home if it occurs within 50 m of the user’s home location. An exposure is annotated with a TIGER road/railway if it occurs within 20 m from that feature. An exposure is annotated as having occurred within a SafeGraph Places POI if the exposure occurs within the polygon defined for the POI. Polygons are provided by the SafeGraph Places database for both fine-grained POIs (for example, individual restaurants) as well as parent POIs (such as hubs). We focus our analysis of fine-grained POIs (Fig. 2c and Extended Data Fig. 3) on the most visited fine-grained POIs, such as full-service restaurants, snack bars, limited-service restaurants (such as fast food) and stadiums (a full list is shown in Fig. 2c). These categories approximately align with those used in previous work[31].

Analysis

Exposure segregation

We define the exposure segregation of a specified geographical area (that is, MSA or county) as the Pearson correlation between the SES of an individual residing in that geographical area and the mean SES of those who they encounter.

$${\rm{E}}{\rm{x}}{\rm{p}}{\rm{o}}{\rm{s}}{\rm{u}}{\rm{r}}{\rm{e}}\,{\rm{s}}{\rm{e}}{\rm{g}}{\rm{r}}{\rm{e}}{\rm{g}}{\rm{a}}{\rm{t}}{\rm{i}}{\rm{o}}{\rm{n}}={\rm{C}}{\rm{o}}{\rm{r}}{\rm{r}}({\rm{S}}{\rm{E}}{\rm{S}},{\bar{{\rm{S}}{\rm{E}}{\rm{S}}}}_{{\rm{e}}{\rm{x}}{\rm{p}}{\rm{o}}{\rm{s}}{\rm{u}}{\rm{r}}{\rm{e}}{\rm{s}}})=\frac{{\rm{c}}{\rm{o}}{\rm{v}}({\rm{S}}{\rm{E}}{\rm{S}},{\bar{{\rm{S}}{\rm{E}}{\rm{S}}}}_{{\rm{e}}{\rm{x}}{\rm{p}}{\rm{o}}{\rm{s}}{\rm{u}}{\rm{r}}{\rm{e}}{\rm{s}}})}{{{\sigma }}_{{\rm{S}}{\rm{E}}{\rm{S}}}{{\sigma }}_{{\bar{{\rm{S}}{\rm{E}}{\rm{S}}}}_{{\rm{e}}{\rm{x}}{\rm{p}}{\rm{o}}{\rm{s}}{\rm{u}}{\rm{r}}{\rm{e}}{\rm{s}}}}}$$

Our metric captures the extent to which an individual’s SES predicts the SES of their immediate exposure network. Thus, in a perfectly integrated area in which individuals encounter others randomly regardless of SES, exposure segregation would equal 0.0. In a perfectly segregated area in which individuals encounter only those of the exact same SES, exposure segregation would equal 1.0. Our primary metric does not upweight repeated exposures to the same person (to avoid overly weighting strong ties such as housemates), although our key findings are robust to doing so (Supplementary Fig. 2).

Exposure segregation nests a classic definition of residential segregation, the neighbourhood sorting index[7], which is equivalent to the Pearson correlation between each person’s SES and the mean SES in their census tract. The neighbourhood sorting index is widely used because it can be calculated directly from census data on the SES of people living in each tract. However, a fundamental limitation of the neighbourhood sorting index as a measure of segregation is that the census tract in which people live is a weak proxy for who they encounter. Census tracts are static and artificial boundaries that do not capture socioeconomic mixing as individuals move throughout the cityscape during work, leisure time and schooling.

We design our exposure segregation metric such that it accommodates any exposure network, and the neighbourhood sorting index is therefore a special case of our metric. Specifically, if exposure segregation is computed for a synthetic exposure network under the unrealistic assumptions that (1) people are exposed only to those in their home census tract; and (2) exposures occur uniformly at random, then it is equivalent to the neighbourhood sorting index (Supplementary Fig. 19). However, constructing such a synthetic exposure network from census tracts has limited applicability to measuring segregation in the real world, because people may also be exposed to more heterogeneous populations as they visit other census tracts for work, leisure or other activities, a phenomenon that we refer to as the visitor effect. Furthermore, even within the home tract, individuals may seek out people of similar SES; we refer to this as the homophily effect. We therefore instead leverage dynamic mobility data from mobile phones to capture the extent of contact between diverse individuals throughout the day, and apply our metric, exposure segregation, to this real-world exposure network. Our analyses reveal that our measure of exposure successfully captures both the visitor effect and the homophily effect (Fig. 2a). An advantage of our definition of exposure segregation is that it allows for direct comparability to the neighbourhood sorting index because both measures are of the same underlying statistical quantity, but differ in their definition of the exposure network. Our results indicate that this choice of exposure network matters; exposure segregation is a stronger predictor of upward economic mobility than the neighbourhood sorting index (Extended Data Fig. 2), and the two metrics are shown to be distinct (Supplementary Fig. 20).

To calculate the exposure segregation of a specified geographical area (that is, MSA or county), we first select the set of all individuals who reside in area \({{\mathcal{V}}}_{A}\subset {\mathcal{V}}\). For example, to calculate exposure segregation for Napa, California (Fig. 1c (top)), \({{\mathcal{V}}}_{A}\) is the 3,707 users with home locations inside the geographical boundary of Napa, CA. Subsequently, for each individual resident of the area \({v}_{i}\in {{\mathcal{V}}}_{A}\), we query the population exposure network (\({\mathcal{G}}=({\mathcal{V}},{\mathcal{E}})\)) for the SES of the set of individuals who they cross paths with \({{\mathcal{Y}}}_{i}\): \(\{{x}_{j}\in {\mathcal{V}}| {e}_{i,j,k}\in {\mathcal{E}}\}\). We then aim to estimate the Pearson correlation between the SES of each individual xi and the (unweighted) mean SES of those to whom they are exposed from all path crossings, \({y}_{i}={\rm{mean}}({{\mathcal{Y}}}_{i})\).

Estimating exposure segregation

Here we first motivate why a ‘naive’ approach to estimating exposure segregation through a sample Pearson correlation on the observed exposure network is problematic (resulting in downwardly biased estimates of exposure segregation). We then elaborate on how we leverage a linear mixed effects model to compute a corrected Pearson correlation, enabling us to obtain unbiased estimates of exposure segregation even in areas where data are sparse.

A naive approach to estimate exposure segregation would be to first compute the observed sample mean SES of individuals who each person is exposed to. Exposure segregation could then be estimated using a sample Pearson correlation:

$${r}_{xy}=\frac{{\sum }_{i=1}^{n}({x}_{i}-\overline{x})({y}_{i}-\overline{y})}{\sqrt{{\sum }_{i=1}^{n}{({x}_{i}-\overline{x})}^{2}{({y}_{i}-\overline{y})}^{2}}}$$

between an individual’s SES (xi) and the sample mean SES of those they are exposed to (yi). This approach is problematic because naively computing such a correlation based on limited data (in counties or MSAs with low population sizes) will result in estimates that are downward biased. To illustrate why naive estimates of exposure segregation are downward biased, imagine that we compute the correlation between a person’s SES and the ‘true’ mean SES of the people who they are exposed to. Now, we add noise to the mean SES values, which represents the noisy mean estimates given limited data. As the noise is increased, the correlation is decreased. Thus, because estimates of each person’s mean SES will be more noisy in geographical areas with less data, there will be a downward bias to naive estimates of the Pearson correlation in these areas.

We instead compute a corrected Pearson correlation, using a linear mixed effects model to accurately estimate exposure segregation: the correlation between a person’s SES and the mean SES of the people they are exposed to. Our linear mixed effects model is an unbiased estimator of the Pearson correlation. We compare the unbiased estimates from our linear mixed-effects model to the downwardly biased sample Pearson correlation estimates in Extended Data Fig. 1.

Our mixed model represents the distribution of datapoints (xi, yij) through the following equation:

$${y}_{ij}=a{x}_{i}+b+{{\epsilon }}_{i}^{(1)}+{{\epsilon }}_{ij}^{(2)},$$

where xi is the SES of person i, yij is the SES of person j who was exposed to person i, a and b are model parameters, \({{\epsilon }}_{i}^{(1)}\) is a person-specific noise term and \({{\epsilon }}_{ij}^{(2)}\) is the noise for each datapoint. Above, the true mean SES of the exposure set for each person is modelled as \(a{x}_{i}+b+{{\epsilon }}_{i}^{(1)}\). Individual exposures yij are then modelled as noisy draws from a distribution centred at this true mean. The Pearson correlation coefficient between person i’s SES and the mean SES of the people they were exposed to is then computed as follows. We assume that xi has a variance of 1 through data preprocessing and that xi is uncorrelated with \({{\epsilon }}_{i}^{(1)}\).

$$\begin{array}{l}{\rm{corr}}({x}_{i},a{x}_{i}+b+{{\epsilon }}_{i}^{(1)})\,=\,{\rm{corr}}({x}_{i},a{x}_{i}+{{\epsilon }}_{i}^{(1)})\\ \,\,\,\,\,=\,\frac{{\rm{cov}}({x}_{i},a{x}_{i}+{{\epsilon }}_{i}^{(1)})}{\sqrt{{\rm{Var}}({x}_{i}){\rm{Var}}(a{x}_{i}+{{\epsilon }}_{i}^{(1)})}}\\ \,\,\,\,\,=\,\frac{{\rm{cov}}({x}_{i},a{x}_{i})}{\sqrt{{\rm{Var}}(a{x}_{i}+{{\epsilon }}_{i}^{(1)})}}\\ \,\,\,\,\,=\,\frac{a}{\sqrt{{a}^{2}+{\rm{Var}}({{\epsilon }}_{i}^{(1)})}}\end{array}$$

We estimate a and \({\rm{Var}}\left({{\epsilon }}_{i}^{(1)}\right)\) by fitting the mixed model using the R lme4 package, optimizing the restricted maximum-likelihood (REML) objective.

Decomposing segregation by time

Each exposure edge (ei,j,k) in our exposure network is timestamped with a time of exposure ti,j,k. This enables us to decompose our overall exposure segregation into fine-grained estimates of segregation during different hours of the day by filtering for exposures that occurred within a specific hour. In Supplementary Fig. 21, we partition estimates of segregation by 3 h windows to illustrate how segregation varies throughout the day (Supplementary Information).

Decomposing segregation by activity

Each exposure edge (ei,j,k) in our exposure network occurs at a specific location lati,j,k, loni,j,k. It is therefore possible to annotate exposures by the fine-grained POI (for example, specific restaurant) that they occurred in, as well as the by the higher-level parent POI (for example, shopping centre) in which the POI was located (see the ‘Annotating exposures’ section). This enables us to decompose our overall exposure segregation into fine-grained estimates of segregation by specific leisure activity. We do so by filtering the network for all exposures that occurred in a specific POI category, and recalculating exposure segregation for the MSA or county using only those exposures. In Fig. 2c, we show the variation in exposure segregation by leisure site, and further explain these variations in Extended Data Fig. 3.

Bridging index

We seek to identify a modifiable, extrinsic aspect of a city’s built environment that may reduce exposure segregation. One promising candidate is the location of a city’s highly visited POIs (that is, hubs). We define a new measure, the bridging index, which measures the extent to which a particular set of hubs (\({\mathcal{P}}\)) may facilitate the integration of individuals of diverse SES within a geographical area (that is, MSA or county). The bridging index measures the economic diversity of the groups that would encounter one another if everybody visited only their nearest hub from \({\mathcal{P}}\), based on the observation that physical proximity significantly influences which hubs individuals visit44,45,46.

The bridging index is computed through two steps (Extended Data Fig. 7):

1. (1)
Cluster all individuals who live in an area (that is, MSA or county residents, \({{\mathcal{V}}}_{A}\)) into K clusters (\({{\mathcal{H}}}_{1},{{\mathcal{H}}}_{2},...,{{\mathcal{H}}}_{K}\)) according to the hub from \({\mathcal{P}}\) closest to their home location. K is the number of hubs in \({\mathcal{P}}\).

2. (2)
The bridging index is computed as the weighted average of the economic diversity (that is, Gini index) of these clusters of people, relative to the area’s overall economic diversity.

$${\rm{B}}{\rm{r}}{\rm{i}}{\rm{d}}{\rm{g}}{\rm{i}}{\rm{n}}{\rm{g}}\,{\rm{i}}{\rm{n}}{\rm{d}}{\rm{e}}{\rm{x}}=\frac{{\rm{W}}{\rm{i}}{\rm{t}}{\rm{h}}{\rm{i}}{\rm{n}}-{\rm{h}}{\rm{u}}{\rm{b}}\,{\rm{e}}{\rm{c}}{\rm{o}}{\rm{n}}{\rm{o}}{\rm{m}}{\rm{i}}{\rm{c}}\,{\rm{d}}{\rm{i}}{\rm{v}}{\rm{e}}{\rm{r}}{\rm{s}}{\rm{i}}{\rm{t}}{\rm{y}}}{{\rm{O}}{\rm{v}}{\rm{e}}{\rm{r}}{\rm{a}}{\rm{l}}{\rm{l}}\,{\rm{e}}{\rm{c}}{\rm{o}}{\rm{n}}{\rm{o}}{\rm{m}}{\rm{i}}{\rm{c}}\,{\rm{d}}{\rm{i}}{\rm{v}}{\rm{e}}{\rm{r}}{\rm{s}}{\rm{i}}{\rm{t}}{\rm{y}}}=\frac{{\sum }_{i=1}^{K}|{{\mathcal{H}}}_{i}|\times {\rm{G}}{\rm{i}}{\rm{n}}{\rm{i}}\,{\rm{i}}{\rm{n}}{\rm{d}}{\rm{e}}{\rm{x}}({{\mathcal{H}}}_{i})}{|{{\mathcal{V}}}_{A}|\times {\rm{G}}{\rm{i}}{\rm{n}}{\rm{i}}\,{\rm{i}}{\rm{n}}{\rm{d}}{\rm{e}}{\rm{x}}({{\mathcal{V}}}_{A})}$$

We illustrate the intuition for our bridging index and how it captures the relationship between home and hub locations in Extended Data Fig. 8. A bridging index of 1.0 indicates that, if everybody visits their nearest hub, each person will encounter a set of people as economically diverse as the overall city they reside in. Thus, a bridging index of 1.0 signifies perfect bridging, that is, even if individuals live in segregated neighbourhoods, hubs are located such that individuals must leave their neighbourhoods and encounter diverse others. On the other hand, a bridging index of 0.0 signifies the opposite extreme; a city with a bridging index of 0.0 is one in which, if everybody visits the nearest hub, each person will encounter only people of the exact same SES.

The economic diversity of each hub \({{\mathcal{H}}}_{i}\) is quantified using the Gini index: \({\rm{G}}{\rm{i}}{\rm{n}}{\rm{i}}\,{\rm{i}}{\rm{n}}{\rm{d}}{\rm{e}}{\rm{x}}({{\mathcal{H}}}_{i})\), a well-established measure of economic statistical dispersion[63] (Extended Data Fig. 7c), although results are robust to choice of economic diversity measure such as using variance instead of Gini index (Supplementary Fig. 14). The bridging index normalizes to the baseline economic diversity observed in the city, enabling direct comparisons between cities.

In our primary analysis, we identify hubs through commercial centres (such as shopping centres and plazas, which are higher-level clusters of individual POIs) because they are associated with a high density of exposures. Specifically, the majority (56.9%) of exposures happen inside of or within 1 km of a commercial centre even though only 2.5% of the land area of MSAs is within 1 km of a commercial centre. We therefore compute our bridging index using the set \({\mathcal{P}}\) of all commercial centres within each MSA. We find that our bridging index strongly predicts exposure segregation (Spearman correlation = −0.78; Fig. 3d). The top 10 MSAs with the highest bridging index are 53.1% less segregated than the 10 MSAs with the lowest bridging index. The bridging index predicts segregation more accurately than population size, racial demographics SES inequality, the neighbourhood sorting index and racial demographics, and is significantly associated with segregation (P < 10−4) after controlling for all aforementioned variables (Extended Data Tables 2 and 3).

Hypothesis testing

Unless otherwise noted, hypothesis tests and CIs were conducted using a bootstrap with 10,000 replications[64]. Steiger’s Z-test was used to compare different predictors of segregation indices, and hypothesis tests for Spearman correlation coefficients were computed using two-sided Student’s t-tests65,66,67. P values were not adjusted for multiple comparisons.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

Nationwide exposure segregation and bridging index measures are available online (http://segregation.stanford.edu). Raw mobility data are not publicly available to preserve privacy. Census data (5 year, 2013–2017 American Community Survey) are available online (https://www.census.gov/programs-surveys/acs). Zillow Rent Estimates are available online (https://www.zillow.com/howto/api/APIOverview.htm). TIGER data (TIGER Roads and Tiger Rails) are available online (https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-geodatabase-file.html). The CoreLogic database is commercially available and may be requested for research use (https://www.corelogic.com/contact/). Individual mobile phone mobility data are not publicly available to preserve privacy, while mobility data aggregated to the CBG level and SafeGraph places data are commercially available and may be requested for research use (https://www.safegraph.com/contact-us).

Code availability

Code is publicly available at GitHub (http://github.com/snap-stanford/exposure-segregation). All analysis was conducted using Python, except for the exposure segregation estimates, which were obtained using a mixed model implemented in R (see the ‘Estimating exposure segregation’ section of the Methods).

References

1. Jacobs, J. The Death and Life of Great American Cities (Random House, 1961).

2. Wirth, L. Urbanism as a way of life. Am. J. Sociol. 44, 1–24 (1938).

3. Milgram, S. The experience of living in cities. Science 167, 1461–1468 (1970).

4. Derex, M., Beugin, M.-P., Godelle, B. & Raymond, M. Experimental evidence for the influence of group size on cultural complexity. Nature 503, 389–391 (2013).

5. Gomez-Lievano, A., Patterson-Lomba, O. & Hausmann, R. Explaining the prevalence, scaling and variance of urban phenomena. Nat. Hum. Behav. 1, 0012 (2016).

6. Stier, A. J. et al. Evidence and theory for lower rates of depression in larger US urban areas. Proc. Natl Acad. Sci. USA 118, e2022472118 (2021).

7. Jargowsky, P. A. Take the money and run: economic segregation in US metropolitan areas. Am. Sociol. Rev. 61, 984–998 (1996).

8. Massey, D. S. & Eggers, M. L. The spatial concentration of affluence and poverty during the 1970s. Urban Aff. Q. 29, 299–315 (1993).

9. Reardon, S. F., Bischoff, K., Owens, A. & Townsend, J. B. Has income segregation really increased? Bias and bias correction in sample-based segregation estimates. Demography 55, 2129–2160 (2018).

10. Schwartz, C. R. Earnings inequality and the changing association between spouses’ earnings. Am. J. Sociol. 115, 1524–1557 (2010).

11. Chetty, R. et al. Social capital: determinants of economic connectedness. Nature 608, 122–134 (2022).

12. Massey, D. & Denton, N. A. American Apartheid: Segregation and the Making of the Underclass (Harvard Univ. Press, 1993).

13. Chetty, R., Hendren, N. & Katz, L. F. The effects of exposure to better neighborhoods on children: new evidence from the moving to opportunity experiment. Am. Econ. Rev. 106, 855–902 (2016).

14. Chetty, R. & Hendren, N. The impacts of neighborhoods on intergenerational mobility II: county-level estimates. Q. J. Econ. 133, 1163–1228 (2018).

15. Chetty, R. et al. Social capital: measurement and associations with economic mobility. Nature 608, 108–121 (2022).

16. Larrabee Sonderlund, A., Charifson, M., Schoenthaler, A., Carson, T. & Williams, N. J. Racialized economic segregation and health outcomes: a systematic review of studies that use the index of concentration at the extremes for race, income, and their interaction. PloS ONE 17, e0262962 (2022).

17. Bor, J., Cohen, G. H. & Galea, S. Population health in an era of rising income inequality: USA, 1980–2015. Lancet 389, 1475–1490 (2017).

18. Do, D. P., Locklar, L. R. & Florsheim, P. Triple jeopardy: the joint impact of racial segregation and neighborhood poverty on the mental health of Black Americans. Soc. Psychiatry Psychiatr. Epidemiol. 54, 533–541 (2019).

19. Brown, J. R., Enos, R. D., Feigenbaum, J. & Mazumder, S. Childhood cross-ethnic exposure predicts political behavior seven decades later: evidence from linked administrative data. Sci. Adv. 7, eabe8432 (2021).

20. Christ, O. et al. Contextual effect of positive intergroup contact on outgroup prejudice. Proc. Natl Acad. Sci. USA 111, 3996–4000 (2014).

21. Hässler, T. et al. A large-scale test of the link between intergroup contact and support for social change. Nat. Hum. Behav. 4, 380–386 (2020).

22. Lee, B. A., Farrell, C. R. & Link, B. G. Revisiting the contact hypothesis: the case of public exposure to homelessness. Am. Sociol. Rev. 69, 40–63 (2004).

23. United Nations Department of Economic and Social Affairs. World Urbanization Prospects: The 2018 Revision Vol. 216 (United Nations Publications, 2018).

24. Ocejo, R. E. & Tonnelat, S. Subway diaries: how people experience and practice riding the train. Ethnography 15, 493–515 (2014).

25. Fischer, C. S. To Dwell Among Friends: Personal Networks in Town and City (Univ. Chicago Press, 1982).

26. Massey, D. S. & Denton, N. A. The dimensions of residential segregation. Soc. Forces 67, 281–315 (1988).

27. Brown, J. R. & Enos, R. D. The measurement of partisan sorting for 180 million voters. Nat. Hum. Behav. 5, 998–1008 (2021).

28. Matthews, S. A. & Yang, T.-C. Spatial polygamy and contextual exposures (spaces) promoting activity space approaches in research on place and health. Am. Behav. Sci. 57, 1057–1081 (2013).

29. Zenk, S. N. et al. Activity space environment and dietary and physical activity behaviors: a pilot study. Health Place 17, 1150–1161 (2011).

30. Moro, E., Calacci, D., Dong, X. & Pentland, A. Mobility patterns are associated with experienced income segregation in large US cities. Nat. Commun. 12, 4633 (2021).

31. Athey, S., Ferguson, B., Gentzkow, M. & Schmidt, T. Estimating experienced racial segregation in US cities using large-scale GPS data. Proc. Natl Acad. Sci. USA 118, e2026160118 (2021).

32. Candipan, J., Phillips, N. E., Sampson, R. J. & Small, M. From residence to movement: the nature of racial segregation in everyday urban mobility. Urban Stud. 58, 3095–3117 (2021).

33. Phillips, N. E., Levy, B. L., Sampson, R. J., Small, M. L. & Wang, R. Q. The social integration of American cities: network measures of connectedness based on everyday mobility across neighborhoods. Sociol. Methods Res. 50, 1110–1149 (2021).

34. Wang, Q., Phillips, N. E., Small, M. L. & Sampson, R. J. Urban mobility and neighborhood isolation in America’s 50 largest cities. Proc. Natl Acad. Sci. USA 115, 7735–7740 (2018).

35. Levy, B. L., Phillips, N. E. & Sampson, R. J. Triple disadvantage: neighborhood networks of everyday urban mobility and violence in US cities. Am. Sociol. Rev. 85, 925–956 (2020).

36. Levy, B. L., Vachuska, K., Subramanian, S. & Sampson, R. J. Neighborhood socioeconomic inequality based on everyday mobility predicts COVID-19 infection in San Francisco, Seattle, and Wisconsin. Sci. Adv. 8, eabl3825 (2022).

37. Xu, Y., Belyi, A., Santi, P. & Ratti, C. Quantifying segregation in an integrated urban physical-social space. J. R. Soc. Interface 16, 20190536 (2019).

38. Abbiasov, T. Do Urban Parks Promote Racial Diversity? Evidence from New York City https://tabbiasov.me/static/85542f4ef5da40a8bf5525be300d3519/NYC_Parks_Diversity.pdf (2020).

39. Chen, M. K. & Rohla, R. The effect of partisanship and political advertising on close family ties. Science 360, 1020–1024 (2018).

40. McPherson, M., Smith-Lovin, L. & Cook, J. M. Birds of a feather: homophily in social networks. Ann. Rev. Sociol. 27, 415–444 (2001).

41. Granovetter, M. S. The strength of weak ties. Am. J. Sociol. 78, 1360–1380 (1973).

42. Sutton, R. Why $400 Sushi Menus Are the New Normal in NYC https://ny.eater.com/2022/1/4/22867230/sushi-prices-omakase-supply-chain-luxury-nyc-restaurants (Eater, 2022).

43. Sutton, R. Masa, Already America’s Priciest Sushi Bar, Now Starts at $1,000 Per Personny.eater.com/2022/3/10/22968257/masa-price-hike-most-expensive-restaurant-sushi-bar-nyc-restaurants (Eater, 2022).

44. Zipf, G. K. The P_1 P_2/D hypothesis: on the intercity movement of persons. Am. Sociol. Rev. 11, 677–686 (1946).

45. Simini, F., González, M. C., Maritan, A. & Barabási, A.-L. A universal model for mobility and migration patterns. Nature 484, 96–100 (2012).

46. Schläpfer, M. et al. The universal visitation law of human mobility. Nature 593, 522–527 (2021).

47. Dietze, P. & Knowles, E. D. Social class and the motivational relevance of other human beings: evidence from visual attention. Psychol. Sci. 27, 1517–1527 (2016).

48. Coston, A. et al. Leveraging administrative data for bias audits: assessing disparate coverage with mobility data for COVID-19 policy. In Proc. 2021 ACM Conference on Fairness, Accountability, and Transparency 173–184 (ACM, 2021).

49. Squire, R. F. What About Bias in the Safegraph Dataset? https://medium.com/safegraph/what-about-bias-in-the-safegraph-dataset-b95b7208893c (SafeGraph, 2019).

50. SafeGraph Dataset https://www.safegraph.com/ (SafeGraph, 2017).

51. Chang, S. et al. Mobility network models of COVID-19 explain inequities and inform reopening. Nature 589, 82–87 (2021).

52. Athey, S., Blei, D., Donnelly, R., Ruiz, F. & Schmidt, T. Estimating heterogeneous consumer preferences for restaurants and travel time using mobile location data. AEA Pap. Proc. 108, 64–67 (2018).

53. Determining Point-of-Interest Visits From Location Data: A Technical Guide to Visit Attribution Technical Report https://www.safegraph.com/guides/visit-attribution-white-paper (SafeGraph, 2020).

54. Corelogic Real Estate Database https://www.corelogic.com/real-estate/ (CoreLogic, 2017).

55. Zillow Rent Zestimate https://www.zillowgroup.com/developers/api/zestimate/zestimates-api/ (Zillow, 2017).

56. Kelton, C. M., Pasquale, M. K. & Rebelein, R. P. Using the North American Industry Classification System (NAICS) to identify national industry cluster templates for applied regional analysis. Reg. Stud. 42, 305–321 (2008).

57. American Community Survey 5-year Estimates https://www.census.gov/data/developers/data-sets/acs-5year.html (US Census Bureau, 2017, accessed 30 August 2017).

58. Statistics of US Businesses Glossary https://www.census.gov/programs-surveys/susb/about/glossary.html (US Census Bureau, 2023).

59. Tiger/Line Shapefiles https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.html (US Census Bureau, 2017, accessed 30 August 2017).

60. Determining patterns SafeGraph docs.safegraph.com/docs/monthly-patterns (SafeGraph, 2020).

61. Madge, C. Public parks and the geography of fear. Tijdschr. Econ. Soc. Geogr. 88, 237–250 (1997).

62. Bentley, J. L. Multidimensional binary search trees used for associative searching. Commun. ACM 18, 509–517 (1975).

63. Dorfman, R. A formula for the Gini coefficient. Review Econ. Stat. 61, 146–149 (1979).

64. Efron, B. & Tibshirani, R. J. An Introduction to the Bootstrap (CRC, 1994).

65. Myers, L. & Sirois, M. J. in Encyclopedia of Statistical Sciences Ch. 12 (Wiley-Interscience, 2004).

66. Steiger, J. H. Tests for comparing elements of a correlation matrix. Psychol. Bull. 87, 245–251 (1980).

67. Corder, G. W. & Foreman, D. I. Nonparametric Statistics: A Step-by-Step Approach (Wiley, 2014).

68. Bailey, M., Cao, R., Kuchler, T., Stroebel, J. & Wong, A. Social connectedness: measurement, determinants, and effects. J. Econ. Perspect. 32, 259–280 (2018).

69. Bailey, M., Farrell, P., Kuchler, T. & Stroebel, J. Social connectedness in urban areas. J. Urban Econ. 118, 103264 (2020).

70. Bailey, M. et al. International trade and social connectedness. J. Int. Econ. 129, 103418 (2021).

71. Kuchler, T., Li, Y., Peng, L., Stroebel, J. & Zhou, D. Social Proximity to Capital: Implications for Investors and Firms Technical Report (National Bureau of Economic Research, 2020).

72. Kuchler, T., Russel, D. & Stroebel, J. Jue Insight: the geographic spread of COVID-19 correlates with the structure of social networks as measured by Facebook. J. Urban Econ. 127, 103314 (2021).

73. Chetty, R., Hendren, N., Kline, P. & Saez, E. Where is the land of opportunity? The geography of intergenerational mobility in the United States. Q. J. Econ. 129, 1553–1623 (2014).

74. The Big Mac Index. The Economist (accessed 29 September 2021); https://www.economist.com/big-mac-index.

75. How much to dine out at a top Michelin-starred restaurant. Chef’s Pencil https://www.chefspencil.com/top-michelin-starred-restaurants-prices/ (2021).

76. Walkability estimates. Walkscore (accessed 30 August 2020); https://www.walkscore.com/methodology.shtml.

Acknowledgements

We thank S. Chang, Y. Roohani, J. Gaebler, S. Goel, A. Chow, E. Jahanparast, S. Manek and A. Shirali for helpful conversations; and N. Singh, R. F. Squire, J. Williams-Holt, J. Wolf, R. Yang and other staff at SafeGraph for mobile phone mobility data and feedback. H.N. was supported by a Stanford Knight-Hennessy Scholarship and the National Science Foundation Graduate Research Fellowship under grant no. DGE-1656518; and E.P. by a Hertz Graduate Fellowship, a Google Research Scholar award, NSF CAREER 2142419, a CIFAR Azrieli Global scholarship, a LinkedIn Research Award, a Future Fund Regrant and the Abby Joseph Cohen Faculty Fund. This research was supported by DARPA under grant numbers HR00112190039 (TAMI) and N660011924033 (MCS); ARO under grant numbers W911NF-16-1-0342 (MURI) and W911NF-16-1-0171 (DURIP); NSF under grant numbers OAC-1835598 (CINES), OAC-1934578 (HDR) and CCF-1918940 (Expeditions); NIH under grant number 3U54HG010426-04S1 (HuBMAP); Stanford Data Science Initiative; Wu Tsai Neurosciences Institute; Amazon; GSK; Hitachi; Juniper Networks; KDDI; and Toshiba. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding entities. We thank the OpenStreetMap contributors, as the maps in Fig. 3 and Supplementary Fig. 65 were generated using OpenStreetMap data (https://www.openstreetmap.org/copyright).

Author information

Author notes

1. These authors contributed equally: Hamed Nilforoshan, Wenli Looi, Emma Pierson

Authors and Affiliations

1. Department of Computer Science, Stanford University, Stanford, CA, USA

Hamed Nilforoshan, Wenli Looi, Nic Fishman, Yiling Chen, John Sholar & Jure Leskovec

2. Department of Computer Science, Cornell Tech, New York, NY, USA

Emma Pierson

3. Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

Blanca Villanueva

4. Institute for Policy Research, Northwestern University, Evanston, IL, USA

Beth Redbird

5. Department of Sociology, Northwestern University, Evanston, IL, USA

Beth Redbird

6. Department of Sociology, Stanford University, Stanford, CA, USA

David Grusky

Contributions

H.N., E.P., B.R., D.G. and J.L. conceived the study concept and design. H.N., W.L., E.P., B.V., N.F., Y.C. and J.S. conducted the computational analysis. All of the authors jointly interpreted the data and wrote the paper.

Corresponding author

Correspondence to Jure Leskovec.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature thanks Eric Hehman and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Unbiased estimates of exposure segregation using our mixed model compared with (downwardly biased) naive estimates using a sample Pearson correlation.

We first compute a gold standard estimate of exposure segregation. We do so by eliminating data sparsity (that is, restricting our analysis to individuals who crossed paths with at least 500 other people) and computing the ‘naive’ Pearson correlation coefficient between each individual’s SES and the mean SES of those with whom they crossed paths (for each MSA). Next, for each person, we randomly downsample their path-crossings to 5, 10, 50, 100, and 200 (x-axis). On this noisy downsampled data, we estimate exposure segregation using both our mixed model (orange) and using the ‘naive’ Pearson correlation (blue). The y axis shows the ratio of these new estimates to the gold standard for each MSA. This analysis reveals that our mixed model enables us to obtain unbiased estimates of exposure segregation, whereas the ‘naive’ Pearson correlation is downwardly biased when observed path-crossings are sparse.

Extended Data Fig. 2 This studies’ exposure network predicts population-scale friendship formation and upward economic mobility outcomes.

We measure the external validity of our definition of exposure by linking our exposure network to outcomes across two gold-standard, large-scale, datasets. We find that at the zip code, county, and MSA-level, our exposure network mirrors population-scale outcomes resulting from dynamic human processes: (a-b) the Facebook social connectedness index[68] measures the relative probability of a Facebook friendship link between a given Facebook user in location i and a given user in location j. FB social connectedness index has been used to study social segregation[69], and has also been linked to economic70,71 and public health outcomes[72]. We reproduce the social connectedness index using our exposure network (\(\frac{\#ExposurePair{s}_{i,j}}{\#Individual{s}_{i}\cdot \#Individual{s}_{j}}\)) at the county (a) and zip code (b) level, and find strong correlations across county pairs (Spearman Correlation 0.85, N = 121, 595, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods) and zip code pairs (Spearman Correlation 0.73, N = 1, 053, 539, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). Furthermore, we find that our exposure network is a stronger predictor of friendship formation than distance (Supplementary Tables 23-24). (c-d) The Chetty et al. intergenerational mobility dataset quantifies upward economic mobility from federal income tax records for each MSA as the mean income rank of children with parents in the bottom half of the income distribution[73]. We find that exposure segregation at the MSA-level (c) correlates to (absolute) upward economic mobility (Spearman Correlation -0.37, N = 379, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods), and does so significantly more strongly (p < 10−4; Two-sided Steiger’s Z-test; see the ‘Hypothesis testing’ section of the Methods) than (d) the conventional segregation measure, neighbourhood sorting index (Spearman Correlation -0.12, N = 379, p < 0.05).

Extended Data Fig. 3 Understanding why exposure segregation varies significantly across leisure sites.

We identify three primary facets of socioeconomic differentiation between POIs which explain the heterogeneous segregation levels of different leisure POIs (Fig. 2c): (a) localization, (b) quantity, and (c) stratification. (a) Localization (average travel distance[30] to the nearest POI of a category) strongly predicts segregation across all POI categories (Spearman Correlation -0.75, N = 17, p < 0.001 Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). POIs which are more locally embedded into neighbourhoods (e.g., religious organizations) are more segregated than POIs which serve multiple neighbourhoods (e.g., stadiums). (b) The quantity of POIs also explains segregation (Spearman Correlation 0.69, N = 17, p < 0.01; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). Leisure activities with more options (e.g., restaurants) have differentiated venues catering to a specific socioeconomic groups (e.g., Michelin-star restaurants) compared to POIs which are small in number and cater to the overall city (e.g., stadiums). (c) Golf courses and country clubs (golf clubs) are an anomaly in that they have a small number of unlocalized POIs, but are highly segregated. We conduct a case study of the top and bottom golf clubs by mean visitor SES in five of the ten largest MSAs. We find that the high segregation of golf clubs is due to extreme stratification between venues; for instance the minimum cost to play at the high-SES golf course in Miami, FL is 11717 × higher than at the lowest-SES golf course. By contrast, the average cost of a MacDonalds Big Mac ($5.65[74]) is only 63 × higher than the average cost of a Michelin 3-star restaurant ($357[75]). Overall, these findings foreshadow the bridging index, which captures POI localization, quantity, and stratification (Extended Data Fig. 8).

Extended Data Fig. 4 Large, dense counties are more segregated.

We compute exposure segregation across 2,829 USA counties (90% of the counties in the USA), excluding counties in which there are less than 50 individuals in our dataset. We find that at the county-level, exposure segregation is also positively correlated with population size (Spearman Correlation 0.45, N = 2829, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods) and population density (Spearman Correlation 0.45, N = 2829, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). These correlations reveal that the association between large, dense urban areas and exposure segregation (Fig. 1d) is not an artifact of city boundaries, and may in fact be an emergent property from dynamics of individuals residing in highly populated, dense geographic areas, which persists across multiple scales of granularity.

Extended Data Fig. 5 At higher levels of scale, spaces in large cities are more differentiated and consequently more segregated:  hubs.

(a-c) We conduct an analysis for a city’s hubs analogous to that for restaurants in Fig. 3c-e for a city’s hubs. We find that higher segregation is driven by an increase in highly differentiated choice of hubs in large cities: (a) Larger MSAs have more hubs, giving residents more options to self-segregate (Spearman Correlation 0.81, N = 382, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). (b) Consequently, hubs in larger MSAs vary more in terms of the mean SES of their visitors (Spearman Correlation 0.58, N = 382, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods) and as a result, (c) exposure segregation within hubs is higher in larger MSAs (Spearman Correlation 0.64, N = 382, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). Overall, this analysis suggests that across multiple levels of scale, large cities offer a greater choice of differentiated spaces targeted to specific socioeconomic groups, promoting everyday segregation in exposures.

Extended Data Fig. 6 At higher levels of scale, spaces in large cities are more differentiated and consequently segregated: home neighbourhoods.

(a-c) Similar to the analysis for restaurants in Fig. 3c-e, we find that higher segregation is driven by an increase in highly differentiated choice of neighbourhoods in large cities: (a) Larger MSAs have more census tracts, giving residents more options to self-segregate (Spearman Correlation 0.97, N = 382, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). (b) Consequently, census tracts in larger MSAs vary more in terms of the mean SES of their residents (Spearman Correlation 0.58, N = 382, p < 10−4; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods) and as a result, (c) both residential segregation (neighbourhood sorting index) and exposure segregation are higher (Spearman Correlations 0.52 and 0.35, N = 382, p < 10−4 and p < 10−4; Two-sided Student’s t-tests; see the ‘Hypothesis testing’ section of the Methods). However, (c) also shows that home tract exposure segregation (green series) rises more slowly with population than conventional segregation (blue series), suggesting that within-home-tract homophily, which increases exposure segregation but not conventional segregation, is not more pronounced in large MSAs. Substantiating this, (d) shows that when home tract exposure segregation is computed using an alternate SES measure so it captures only within-home-tract-homophily, it is not higher in large MSAs (Spearman Correlation -0.01, N = 382, p > 0.1; Two-sided Student’s t-test; see the ‘Hypothesis testing’ section of the Methods). The alternative SES measure is computed by subtracting the mean SES in each census tract. Overall, this analysis suggests that the higher home tract segregation in large MSAs is driven by people’s greater choice of neighbourhoods of varying SES in which to live, but not by a greater tendency to cross paths homophilously within their own neighbourhood.

Extended Data Fig. 7 Computing bridging index.

Illustration of our analytical pipeline for calculating the bridging index. (a) Bridging index is computed from the locations and number of POIs in the MSA which are expected to be hubs of exposure (that is, frequently visited POIs), as well the locations and SES values of all homes within MSA boundaries. We intentionally develop bridging index without using mobility data, with the intention of identifying a modifiable extrinsic aspect of a city that can be intervened on to impact mobility patterns and decrease exposure segregation. (b) In order, we (1) cluster all homes by nearest hub (using straight line distance from home to hub), partitioning all homes into K clusters, where K is the number of hubs in the MSA (2) compute the weighted average economic diversity (i.e., Gini index) of the clusters, normalized by the overall economic diversity of the MSA to allow for comparisons between different MSAs of varying baseline levels of economic diversity (Extended Data Table 1). (c) The graphical definition of Gini index is provided, which is a standard measure of economic dispersion[63]. Results are robust to the definition of economic diversity, and hold true when using variance in SES instead of Gini index (Supplementary Fig. S14).

Extended Data Fig. 8 Understanding the determinants of the bridging index.

The bridging index is a single metric which captures three important factors of built environment (see Supplementary Fig. S13 for contributions of these factors to explaining exposure segregation): (1) The locations of hubs—if hubs are located in between diverse neighbourhoods, the bridging index will be high as hubs will bridge together diverse individuals. (2) The numberof hubs—as number of hubs decreases, bridging index increases (e.g if there is only 1 hub in a city, bridging index will be 1.0 as all individuals are unified by a single hub) (3) Residential segregation, i.e., the locations of homes and their associated SES—as residential segregation decreases we can expect that individuals residing near each hub will be more diverse. This figure builds intuition for how the bridging index may vary for a single simulated city, consisting of highly segregated neighbourhoods. We hold residential segregation (3) constant, and vary the location (1) and number (2) of hubs across panels (a), (b), (c), (d), in order of increasing bridging index. Note that the bridging index in (c) is substantially higher than the bridging index in (b), because hubs in (c) are better positioned to bridge diverse neighbourhoods—even though the number of hubs remains constant.

Extended Data Table 1 Population size is significantly associated with exposure segregation, after controlling for MSA income inequality (Gini index), political alignment (% Democrat in 2016 election), racial demographics (% non-Hispanic White), mean SES, walkability (Walkscore[76]), commutability (% of residents commuting to work), and residential segregation (neighbourhood sorting index)

Extended Data Table 2 Bridging index is significantly associated with exposure segregation, after controlling for MSA population size, number of hubs, income inequality (Gini index), political alignment (% Democrat in 2016 election), racial demographics (% non-Hispanic White), mean SES, walkability (Walkscore[76]), commutability (% of residents commuting to work), and residential segregation (neighbourhood sorting index)

Extended Data Table 3 Bridging index strongly predicts exposure segregation

Supplementary information

Supplementary Information

Supplementary Figs. 1–66 and Supplementary Tables 1–24.

Reporting Summary

Peer Review File

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Nilforoshan, H., Looi, W., Pierson, E. et al. Human mobility networks reveal increased segregation in large cities. Nature 624, 586–592 (2023). https://doi.org/10.1038/s41586-023-06757-3

- Received: 27 October 2022

- Accepted: 17 October 2023

- Published: 29 November 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06757-3

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Mobility patterns are associated with experienced income segregation in large US cities

Article Open access 30 July 2021

Esteban Moro, Dan Calacci, … Alex Pentland

Uncovering the socioeconomic facets of human mobility

Article Open access 21 April 2021

Hugo Barbosa, Surendra Hazarie, … Gourab Ghoshal

Quantifying social segregation in large-scale networks

Article Open access 19 April 2022

Bjørn-Atle Reme, Andreas Kotsadam, … Jo Thori Lind

The myth of cosmopolitan cities: why large urban areas are more segregated

Nature Research Briefing 29 Nov 2023

Article Open access Published: 13 December 2023

Indigenous Australian genomes show deep structure and rich novel variation

Matthew Silcocks,

Ashley Farlow,

Azure Hermes (Gimuy Walubara Yidinji),

Georgia Tsambos,

Hardip R. Patel,

Sharon Huebner,

Gareth Baynam,

Misty R. Jenkins (Gunditjmara),

Damjan Vukcevic,

Simon Easteal,

Stephen Leslie &

The National Centre for Indigenous Genomics

Show authors

Nature volume 624, pages 593–601 (2023)

- 13k Accesses

- 1 Citations

- 194 Altmetric

- Metrics details

Abstract

The Indigenous peoples of Australia have a rich linguistic and cultural history. How this relates to genetic diversity remains largely unknown because of their limited engagement with genomic studies. Here we analyse the genomes of 159 individuals from four remote Indigenous communities, including people who speak a language (Tiwi) not from the most widespread family (Pama–Nyungan). This large collection of Indigenous Australian genomes was made possible by careful community engagement and consultation. We observe exceptionally strong population structure across Australia, driven by divergence times between communities of 26,000–35,000 years ago and long-term low but stable effective population sizes. This demographic history, including early divergence from Papua New Guinean (47,000 years ago) and Eurasian groups[1], has generated the highest proportion of previously undescribed genetic variation seen outside Africa and the most extended homozygosity compared with global samples. A substantial proportion of this variation is not observed in global reference panels or clinical datasets, and variation with predicted functional consequence is more likely to be homozygous than in other populations, with consequent implications for medical genomics[2]. Our results show that Indigenous Australians are not a single homogeneous genetic group and their genetic relationship with the peoples of New Guinea is not uniform. These patterns imply that the full breadth of Indigenous Australian genetic diversity remains uncharacterized, potentially limiting genomic medicine and equitable healthcare for Indigenous Australians.

Similar content being viewed by others

Main

The Indigenous populations of Australia remain poorly represented in sequencing panels and clinical databases. Their inclusion is warranted on the grounds of equity and their unique demographic history. Indigenous Australians probably descend from an early dispersal of humans across Asia[3], inheriting substantial ancestry from extinct hominin groups1,4,5. Previous DNA studies have identified novel variation[6] and inferred a long history of geographical regionalism in Australia[7]. An earlier whole-genome sequencing study inferred a sudden separation from Papuans 25–40 thousand years ago (ka) and divergence within Australia occurring 10–32 ka (ref. [1]). Importantly, all 83 participants in the study were Pama–Nyungan language speakers, a language family that is widespread across Australia despite its relatively recent origin (estimated at 6 ka)[8], possibly accounting for the lack of strong discernible structure[1]. It is estimated that another 27 language families[9], largely restricted to the Top End and Kimberley region, are unrepresented in genomic data. Linguistic variation is often correlated with patterns of genetic variation[10], supporting the inclusion of speakers of these languages in genomics studies.

If limited population structure remains after more representative geographical and language group sampling, a common set of genomic tools and reference panels will be sufficient to inform medical research and clinical practice. Alternatively, previously undocumented structure, due to patterns of migration, isolation and population size change, may indicate the poor suitability of such panels and support wider sampling to capture the full distribution and diversity of common and rare alleles.

Such patterns can be explored by quantifying the levels of novel and shared variation relative to other human populations and by applying population genetic models to determine structure and its causes. Both approaches require adequate sampling within communities and the inclusion of communities that capture the breadth of the underlying genetic diversity.

The NCIG collection

The Australian National University holds more than 7,000 biospecimens collected between the 1960s and 1990s from about 40 Indigenous communities (Supplementary Note 1). A panel of leading Aboriginal and Torres Strait Islander Australians recommended the collection be placed under Indigenous-majority custodianship, leading to the establishment of the National Centre for Indigenous Genomics (NCIG) in 2016[11]. The primary role of NCIG is to engage with Indigenous communities on the existence and nature of the collection, extend and promote its use for research and ensure that research is done with appropriate personal consent and community engagement (Methods).

During recent community engagement, 159 community members provided new blood or saliva samples under modern consent and ethics protocols. This study analyses genetic data from these Indigenous Australians from four environmentally diverse regions across northern and central Australia, including tropical savannah and rainforest, remote islands and desert. (Clearly these environments will have varied over the many millennia Indigenous Australians have lived on the continent). This is a large and purposefully diverse collection of genomic data from Indigenous Australians.

The cohort includes 59 individuals from the Tiwi Islands. The Tiwi people experienced a long period of isolation from mainland Australia[12] and speak a linguistic isolate unrelated to the Pama–Nyungan languages spoken by the other three communities involved. Included are 33 people from the community of Wurrumiyanga on Bathurst Island, 20 from Milikapiti and six from Pirlangimpi on Melville Island. This is about 3% of the current population of the islands (around 2,000). The cohort also includes 48 individuals from the community of Yarrabah on the traditional lands of the Gunggandji and Mandingalbay Yidinji. The Yarrabah Aboriginal Mission, established in 1892, was used as a settlement for displaced Indigenous people from across Queensland. In 1938, 43 different tribal groups were represented in Yarrabah[13]. The cohort contains 14 people from the Central Desert community of Titjikala, comprising of members of the Southern Arrernte, Yankunytjatjara, Luritja and Pitjantjatjara. Finally, there are 38 individuals from the community of Galiwin’ku on Elcho Island. Established in 1942, the community comprises members of 30 closely related clan groups (Yalu team Galiwin’ku, personal communication).

DNA was extracted from either blood or saliva and Illumina sequenced to high coverage (minimum 30×, median 42×; see Methods and Supplementary Note 2). Variants were called jointly and phased with 60 previously sequenced individuals from geographically adjacent populations (25 men from the highlands of Papua New Guinea (PNG) drawn from five different language groups[1] and 35 men from 11 regions of the Bismarck Archipelago of PNG in Island Melanesia[5]).

Genetic ancestry in the collection

We emphasize that genetic ancestry proportions may or may not align with identity and that all communities worldwide have varying degrees of shared ancestry. Nonetheless, we seek to focus on genetic ancestry that is Indigenous Australian in origin. Thus, our cohort was combined with the 1000 Genomes Project samples[14] (hereafter 1000 Genomes), and we applied standard algorithms to identify genomic regions with ancestry other than Indigenous ancestry (Methods and Supplementary Note 2). We find that 100 of 111 individuals from Titjikala, Galiwin’ku and Tiwi have only Indigenous ancestry (Extended Data Fig. 1a). By contrast, consistent with the history of the community, all Yarrabah individuals have an appreciable degree of European, East Asian and/or putative Melanesian ancestry (mean 41%, range 11–73%). Notably, and consistent with known sex-specific demographic patterns1,15, all Australian individuals have a mitochondrial lineage belonging to a previously documented Indigenous Australian haplogroup (see ‘Mitochondrial diversity’ section).

To avoid genomic regions of non-Indigenous ancestry confounding analyses, local ancestry was inferred along each haplotype on the basis of a reference panel of individuals thought to be unadmixed from Australia, PNG, Eurasia and Africa. Genomic regions were masked within an individual if one or both haplotypes were inferred to be of non-Indigenous ancestry: that is, neither Australian nor Papuan (see ‘Ancestry inference’ in Methods and Supplementary Note 2). Ten individuals from Tiwi showed patterns of polymorphism and clustering consistent with having at least one recent ancestor from an Indigenous community other than Tiwi (Supplementary Note 2). Unless otherwise stated, all analyses were performed on this ancestry-masked dataset, filtered to remove these ten Tiwi individuals and first- and second-degree relatives, leaving 89 individuals (34 Tiwi, 31 Yarrabah, 17 Galiwin’ku, 7 Titjikala).

The size of this collection, its geographical distribution and the limited non-Indigenous ancestry is notable compared with previous studies1,16,17. This allowed for characterization of novel and shared genetic variation at the individual and population levels and inference of the demographic forces that have generated these patterns.

Australian variation in a global context

The suitability of current reference databases for genomics involving Indigenous Australians depends on how well they capture variation in these populations. Of the 9.9 million single-nucleotide variants (SNVs) observed across all 159 individuals after ancestry masking, 3.4 million (34%) are not present in either the 1000 Genomes[18] or the Human Genome Diversity Project (HGDP)[19] (Extended Data Table 1). For comparison, only 10% of SNVs observed in the analysed Papuan individuals are absent from both datasets, probably because of the Papuan samples in the HGDP. Of the variants seen in the Australian cohort, 26% are not observed in either PNG individuals or the Genome Aggregation Database (gnomAD) release 3.1 (which has 76,000 samples, including the 1000 Genomes and HGDP)[20]. This is important as rarity in gnomAD is one metric used to prioritize potentially pathogenic variants for clinical diagnostics. Out of all variants observed, 2.1 million are restricted to a single Indigenous Australian population sample. Thus, given the limitation of current sampling, between 6.3% and 8.7% of SNVs in each of these four population samples are not observed elsewhere.

To compare the proportions of novel and geographically restricted variation across populations, we analysed equal subsamples of five unrelated individuals from each of the 32 populations in our cohort and the 1000 Genomes (Fig. 1a,b). This ensured that the smallest sample, Titjikala, was included. As all individuals from Yarrabah have some non-Indigenous ancestry, the five with the least missing data after ancestry masking were selected (for analyses without subsampling or ancestry masking, see Supplementary Fig. 1). The observations below hold for larger subsamples of 15 and 25 individuals per population (Supplementary Fig. 1).

Fig. 1: Variant characteristics across populations.

a,b, Per-population count (a) and proportion of total variation (b) for biallelic SNVs across four classes of sharing for samples of five individuals per population (samples of 15 and 25 in Supplementary Fig. 1). Bars: values for single representative populations. Lines: range for other continental populations. Sharing defined relative to all 26 populations of the 1000 Genomes and the six Oceanic populations considered here. c, Distribution of minor allele count within each population sample (restricted to five as above). Minor allele defined by pooling the five individuals from each of the 32 populations. d–f, Per-individual count of heterozygous sites (d), homozygous amino acid substitutions with predicted functional consequence (e) (SIFT score less than 0.05 (ref. [24])) and proportion of the genome in extended homozygosity (f) (ROH more than 1 Mb). Outside Oceania, values are for the population indicated, with continental distributions summarized by black box plots (median (line), upper/lower quartiles (box) and 1.5× interquartile range (whiskers)). Values before masking (dashes) and rescaled after masking (circles) are shown for individuals with more than 5% ancestry other than Indigenous ancestry. ROH estimated from unmasked data and therefore not rescaled. ROH values for individuals with more 5% ancestry other than Indigenous ancestry shown as dashes. (Tiwi n = 48, Galiwin’ku n = 38, Titjikala n = 13, Yarrabah n = 45, PNG (HL) n = 25, PNG (Is.) n = 35, YRI n = 108, STU n = 102, GBR n = 91, CHS n = 105, PEL n = 85). g, Variant discovery with increasing sample size per population, averaged (ten replicates). Yarrabah and PNG (Is.) excluded because of missing data after ancestry masking. h, Novel variant discovery per continent after sampling 80 individuals from each of the other continents, averaged (ten replicates). 1000 Genomes codes: YRI, Yoruba, Africa; PEL, Peruvian, America; STU, Sri Lankan, South Asia; GBR, British, Europe; CHS, Southern Han Chinese, East Asia.

Consistent with previous studies21,22, total autosomal variation declines with distance from sub-Saharan Africa. Indigenous Australians and Papuans have the least total variation of any population analysed here, with the largest deficit for variation shared across some but not all continents (Fig. 1a,b), consistent with previous reports showing that the separation of Australians and Papuans predates that of all other populations outside Africa[1]. Indigenous Australians have the highest count of variation that is either private to population or private to continent outside Africa (Fig. 1a,b and Supplementary Fig. 1). This ranges from 7.3% to 9% of SNVs in Oceania, with the next highest (6.1%) the JPT (Japanese in Tokyo, Japan), in East Asia. Interestingly, variation occurs less often as singletons in Oceania, particularly among Tiwi people, with the minor allele frequency spectrum showing more variation at a higher frequency within a population sample than seen in populations of other continents (Fig. 1c).

Indigenous Australians and Papuans have the lowest heterozygosity worldwide (Fig. 1d). Within the region, on average the Tiwi had the lowest genetic diversity and Yarrabah the highest (both before and after the ancestry masking (Fig. 1d)), reflecting the diverse origins of the latter community.

The high levels of population- and continent-private variation in Oceania extend to polymorphisms of potential functional significance. Our cohort lacks phenotypes, so associating genetic variation with diseases relevant to Indigenous communities is impractical, although some observations may be made. Considering coding variation in 32 genes associated with type 2 diabetes[23], we find 51 non-synonymous variants in Galiwin’ku (other groups are similar). Of these, five are either population-private or private to Oceania. These values are typical for equal sample sizes from Europe, Asia and the Americas (Supplementary Note 3). Genome-wide, people in Oceania also have typical numbers of variants of predicted functional consequence on the basis of sequence constraint (SIFT[24] and PolyPhen[25], Supplementary Table 1). However, genomes of people from Oceania have fewer variants annotated as pathogenic or likely pathogenic in the clinical database ClinVar[26] (Supplementary Fig. 1f and Supplementary Table 1), no doubt because of ascertainment bias in ClinVar. Averaged across each Oceanic population sample, we observe 104 variants (median, range 97–108) designated pathogenic or likely pathogenic in ClinVar (0.00225% of variants), whereas the European samples average 184 (median, range 174–202, 0.00313% of variants).

Of relevance to clinical interpretation of predicted functional variation, Indigenous Australians and Papuans have the highest proportions on average of their genomes in runs of homozygosity (ROH; Fig. 1f and Extended Data Fig. 1b,c). Individual values are typically more extreme than those of the Indigenous American peoples (PEL) from Peru, a largely unadmixed population with a low long-term effective population size[14] and reduced heterozygosity consistent with serial founder events[2]. For example, Tiwi genomes typically exceed 10% extended homozygosity, three times that of Indigenous American peoples (Fig. 1f) and ten times that of Eurasian populations. This extended homozygosity is consistent with elevated background relatedness, probably because of a low long-term effective population size, rather than consanguinity, which is often observed in population isolates[27] (Extended Data Fig. 1b). Variation with predicted functional consequence more likely occurs in the homozygous state in Oceania than elsewhere (Fig. 1e).

Sample size and variant discovery

The distribution of variation will affect studies of disease genetics in Indigenous populations. Although the engagement of communities with genomic studies is their choice[28], our results inform the design of sampling approaches to maximize recovered diversity. To understand the sample size required to adequately capture common variation in Indigenous Australian populations, we calculated variant discovery with progressively increasing sample size[29]. Despite having the highest levels of population-private variation outside Africa (Fig. 1a), the discovery of this variation saturates at much lower sample sizes than for populations on other continents (Fig. 1g). Although the 1000 Genomes populations continue to reveal more variants with increasing sample size, partly because of the steady accumulation of rare variants (including singletons), the number of new variants added by each additional genome of individuals from Oceania diminishes more rapidly. This is consistent with the skewed allele frequency spectra in these samples (Fig. 1c) and indicates relatively small effective population sizes.

Even at small sample sizes, individuals from Oceania have substantial uncharacterized variation. After sampling 80 individuals from each of the other continents, we tested how much novel variation was recovered when sampling within each continent (Fig. 1h). This revealed rates of novel variant discovery in Oceania similar to those in East and South Asia, up to a sample of around 30, much greater than the rates of either Europe or the Americas (this is probably affected by admixture of people from Europe with those from the Americas).

Population structure

Although the sample sizes required for an Indigenous Australian genomic reference panel are probably small, the breadth of communities to include will depend on population structure across the continent. Population structure arises when non-random mating produces systematic differences in allele frequencies between subsets of a larger population. The nature and strength of such structure is typically a consequence of demographic processes such as isolation, population divergence times, historic effective population sizes and migration rates. Understanding structure is fundamental for studies of demography and disease30,31.

Applying a range of methods, we detect structure and classify individuals into clusters that coincide extensively with their geographical origin (Fig. 2, Extended Data Figs. 2–4 and Supplementary Note 4). More precisely, hierarchical clustering of pairwise outgroup F_3 statistics (Fig. 2b), ADMIXTURE[32] (Fig. 2c) and fineSTRUCTURE[33] (Fig. 2f) cluster individuals. Elsewhere, the geographical labels coincide strongly with the discriminating measures of the analysis. In each analysis, the overwhelming majority of individuals are assigned to ‘correct’ (geographically defined) groups, and for the Tiwi (uniform manifold approximation and projection (UMAP)[34] and fineSTRUCTURE) and PNG Highlands (HL) (UMAP), groups are assigned at fine geographical scales of as little as tens of kilometres. Except for four individuals from Titjikala and Yarrabah, hierarchical clustering and ADMIXTURE-inferred groups coincide with geographical labels, and fineSTRUCTURE is concordant for all individuals analysed (Fig. 2f and Extended Data Fig. 4). These methods infer a bifurcation between Australian and Papuan groups, followed by the divergence of the Tiwi—the only Australian group to speak a non-Pama–Nyungan language (Fig. 2b,c and Extended Data Fig. 3). Rare allele and identity-by-descent (IBD) tract sharing between individuals from the same region is higher than for individuals from different groups, revealing strong within-sample homogeneity (Fig. 2d).

Fig. 2: Population structure.

a, Location and sample size for all Australian and Papuan samples. b, Hierarchical clustering of unrelated individuals on the basis of pairwise outgroup F_3 statistic values. Colour corresponds to sampling location. c, ADMIXTURE-inferred ancestry for unrelated individuals allowing seven clusters, ordered according to sampling location. Colour was assigned to each cluster post hoc on the basis of the scheme in a and the majority membership of each cluster. d, Pairwise sharing of rare alleles (above diagonal) and IBD (below diagonal) tracts among all individuals. Counts were rescaled according to the proportion of the genome missing due to ancestry masking in each pairwise comparison. Comparisons between first- and second-degree relatives are indicated in red. e, UMAP clustering of unrelated individuals on the basis of minor allele frequency-corrected COV distances, reduced to the first ten components by MDS. Box expands the positions of Tiwi Island individuals. f, Clustering of Tiwi individuals on the basis of co-ancestry values estimated using fineSTRUCTURE run on all unrelated and unadmixed samples (see Extended Data Fig. 4a for the full tree). Light blue (Bathurst Island) and dark blue (Melville Island) indicate sampling location, and yellow and grey indicate cluster membership.

The complex population structure shows that Indigenous Australians form neither a single genetic population nor one with PNG. Rather, we observe a striking, previously undescribed pattern of regional differentiation. For context, we apply several of the same methods to the 1000 Genomes continental groups (Extended Data Figs. 5 and 6). Although these have undergone different demographic events, including expansions and large-scale admixture19,35, subsamples have similar geographical separation. ADMIXTURE and hierarchical clustering do not group Europeans, East Asians and South Asians with the same accuracy as populations in Oceania (Extended Data Fig. 6), which have a structure more like that of sub-Saharan Africans. Pairs of individuals in Eurasian populations typically share fewer than five IBD tracts longer than 1.5 cM (Extended Data Fig. 5), an order of magnitude smaller than typical in Australian populations (Fig. 2d).

Pairwise fixation index (FST) estimates for populations in Australia compared with those between Simons Genome Diversity Panel (SGDP) populations (Extended Data Fig. 7) further support a scale of population structure in Australia that is among the strongest seen between human populations sampled from the same continent. Taken together, our results demonstrate that it is vital to broadly sample Indigenous Australian and Papuan populations for clinical applications and for characterizing the full spectrum of human genetic variation.

Relationship to PNG

The strength of structure within Australia and to PNG shows that samples from PNG (which contribute to gnomAD via the HGDP collection) are an inadequate reference for variation in Australia. To understand whether the relationship to PNG is uniform across all Australian populations, we use F statistics[36], measures of shared genetic drift, to explore potentially subtle differences in allele sharing with PNG.

We find significant differences (Kruskal–Wallis omnibus test) between Australian populations in their shared drift with PNG (Fig. 3a; outgroup F_3 statistics). Samples from Titjikala share less drift with PNG than those from Tiwi or Galiwin’ku and most samples from Yarrabah, and the Titjikala samples are not derived from the same distribution as the other samples (pairwise Mann–Whitney U-tests; Extended Data Fig. 8a). Yarrabah individuals have highly variable F_3 statistics, correlated with the degree of recent PNG-related ancestry inferred in each genome (Fig. 3a; Spearman’s correlation coefficient permutation test P = 0.017). Although several scenarios, explored below, could result in these patterns, this excludes a single division of ancestral Australian and Papuan populations without subsequent genetic interactions.

Fig. 3: Historical relationships between Australian and PNG populations.

a, Top, shared genetic drift between populations estimated by outgroup F_3 statistics of the form F_3 (Yoruba; PNG, X), where X is an Australian individual. Higher values indicate greater shared genetic drift with PNG. Individuals are rank-ordered by F_3 value within populations, with block jackknife-estimated standard errors shown as vertical bars. The range of F_3 values for individuals in the Tiwi and Galiwin’ku population samples is indicated by horizontal shading. Bottom, the proportion of Papuan global ancestry (after masking) estimated by RFMIX for the same individuals. These per-individual metrics include related individuals. Sample sizes: Tiwi n = 48, Galiwin’ku n = 38, Titjikala n = 13, Yarrabah n = 45. b, Z-scores derived from F_4 statistics of the form \({F}_{4}^{({\rm{T}})}\)(Asia-Y, Yoruba; Australia-X, Titjikala), where Asia-Y is a Eurasian or Oceanic population sample from SGDP and Australia-X is either the Galiwin’ku or Tiwi Islands sample. Z-score values greater than 3 provide statistically significant evidence that population Asia-Y shares more genetic drift with Tiwi/Galiwin’ku than with Titjikala, and these populations are marked with an asterisk. The per-individual metrics include related individuals.

We calculate \({F}_{4}^{({\rm{T}})}\) statistics of the form \({F}_{4}^{({\rm{T}})}\) (YRI, PNG; Australia-X, Australia-Y) to formally test for a non-cladistic relationship between Australian and PNG populations. We reject the null hypothesis that Australian populations form a clade (that is, are equally related) with respect to PNG for every combination pairing Titjikala with another Australian group (Extended Data Fig. 8b), confirming the three northern populations share more genetic drift with PNG, contrary to previous reports1,6. Although this and previous studies infer recent PNG or Torres Strait Island-related ancestry in North Queensland (here Yarrabah)1,17,37, we find no evidence of recent admixture from a PNG-related source population into Tiwi or Galiwin’ku (Extended Data Fig. 8c). By expanding the F_4 analysis to include more Asian and Oceanic populations (SGDP[6]), we rule out common admixture from an external source population (for example, from Island Southeast Asia) into both PNG and Tiwi and/or Galiwin’ku, explaining the elevated shared drift with PNG (Fig. 3b). The remaining plausible demographic scenario is an extended period of genetic interaction between the ancestral populations of PNG and northern Australia once structure began to form within Australia. Differential ancestry from extinct hominin groups may also have affected these patterns but was not investigated.

To assess whether shared drift with Australian populations is uniform across PNG, we calculate outgroup F_3 statistics using genotype data for a larger collection of individuals from PNG (Extended Data Fig. 9). The values are uniformly higher for Tiwi than Titjikala across all regions, showing that genetic interaction between northern Australia and PNG ceased before structure developed within PNG or that any early structure within PNG was erased by later migrations. We note that this analysis only considers groups from the east of the island of New Guinea.

Historical relationships in Australia

The relative importance of the heterogeneous relationships to PNG depends largely on demographic parameters, including effective population sizes, split times and migration rates within Australia. We apply an approach combining efficient simulation of genetic data[38] with approximate Bayesian computation (ABC)[39] to evaluate evidence for each of seven plausible phylogenetic topologies. Modelling several migration parameters, we assess the contribution from PNG to each Australian population and between Australian populations over time (Fig. 4a,  Methods and Supplementary Note 5). The topology with the most support (scenario 4, Fig. 4a) and the most-supported combination (scenarios 4–6, Fig. 4a) have the Tiwi as an outgroup to the other Australian groups, supporting a division on the basis of language family rather than geographical distance. This was confirmed by an alternative approach, AdmixtureBayes[40] (Supplementary Fig. 5; 37.6% of sampled trees have the Tiwi as an outgroup).

Fig. 4: Historical relationships and processes that have shaped genomic variation in the sample.

a, Seven plausible population histories were included in ABC simulations, in which effective population sizes and migration rates were allowed to vary. Also shown is the evidence for each scenario or group of scenarios. b, Parameter estimates for split times and effective population sizes for the most likely single scenario, scenario 4 (see Supplementary Figs. 2–4 for parameter distributions). BF, Bayes factor; GAL, Galiwin’ku; Ne, effective population size; P, posterior probability; PNG, Papua New Guinea (HL); TIJ, Titjikala; TIW, Tiwi Islands; YAR, Yarrabah.

Both methods support Galiwin’ku as an outgroup to Titjikala and Yarrabah (32.7% of AdmixtureBayes-sampled trees). However, our ABC analysis cannot rule out some alternatives (scenarios 5 and 6), and AdmixtureBayes supports a star-like consensus topology with extremely short internal branch lengths and long terminal branches (Supplementary Fig. 5). We formally tested whether Tiwi and Galiwin’ku, the two geographically closest communities, form a clade with respect to the other Australian groups (scenarios 1 and 2) but found little evidence to support this (Fig. 4a).

On the basis of the best-supported topology (Fig. 4b), we used our ABC method to estimate split times (Supplementary Table 2 and Supplementary Fig. 2), effective populations sizes (Supplementary Table 3 and Supplementary Fig. 3) and migration rates (Supplementary Fig. 4). We infer that the split between Indigenous Australians and Papuans occurred 1,636 generations ago (47 ka, highest 95% posterior density interval 27–64 ka). This is older than the previous estimate of 37 ka from autosomal data[1] but consistent with estimates from mitochondrial DNA[7]. This ancestral Australian population existed for 12,000 years with a small but statistically well-supported effective population size of around 2,000 (median; Supplementary Fig. 3 and Supplementary Table 3), followed by the relatively rapid separation of the ancestral populations of Tiwi (35 ka) and Galiwin’ku (31 ka) and a Titjikala–Yarrabah split at 26 ka.

The early and rapid division of Australian groups inferred via ABC, the star-like consensus topology inferred by AdmixtureBayes and the Multiple Sequentially Markovian Coalescent-2 (MSMC2) analysis presented below imply that the history of these populations probably involved a complex period of overlapping and incomplete isolation. However, once isolation was established, the methods infer limited migration between groups, although we caution that there is poor inference of historic migration rates with ABC (Supplementary Fig. 4), and no admixture events were inferred in the 15 top-ranked AdmixtureBayes trees (Supplementary Note 5).

Notwithstanding these findings, several lines of evidence (Supplementary Note 5) are consistent with recent Papuan or Melanesian ancestry in individuals from Yarrabah. Explicitly modelling this in the last three to seven generations gave strong support for a 1.8% contribution from PNG or a PNG-proximal population into the current Yarrabah population (Supplementary Table 2 and Supplementary Fig. 4). With this exception, combining the evidence presented here with the strong population structure observed above indicates that long-term migration between populations was limited relative to other global populations. We note that these inferences, on the basis of genetics, can also be strongly informed by community knowledge and history.

Effective population size

Using the ABC model, we infer that the Tiwi, Galiwin’ku and PNG populations underwent historic changes in effective population sizes, with strong support for an extended period of large effective population sizes, 10,000 for Galiwin’ku and 7,000 for Tiwi, before undergoing a strong reduction (Supplementary Fig. 3). The approach gives poor resolution on the time of these events, so we apply two methods that leverage historic recombination events to infer effective population size: over the last few hundred generations (IBDNe)[41] and deeper in time (MSMC2)[42].

The past 6,000 years are characterized by small but stable effective population sizes ranging from around 10,000 for Yarrabah (likely inflated by the diverse origins of this community) down to 1,500 for the Tiwi Islands, a value consistent with historical surveys of the census population size[12] (IBDNe; Fig. 5a, Supplementary Fig. 6 and Supplementary Note 6). We infer a marked decline in population sizes over the past few hundred years, although this is less evident for Titjikala. This contrasts with Eurasian populations, which have had steady population growth over the past 8,000 years, with a rapid increase in the past 1,000 (refs. 19,41,43).

Fig. 5: Effective population sizes and population isolation.

a, Mean effective population size estimates using the IBDNe algorithm. Shading indicates 95% bootstrap confidence intervals. b, Effective population size estimates for Australian and PNG (HL) populations inferred using MSMC2 from eight phased haplotypes (four individuals) per population. The line and shading are the mean and s.e.m. of five replicates randomly selected from each population sample. Grey bar indicates the Last Glacial Maximum (21 ± 3 ka). c, rCCRs for all 45 possible population pairs (5 Australian + 5 PNG (HL)) estimated with MSMC2. Each line represents the mean rCCR of ten selected sets of eight phased haplotypes (2 haplotypes × 2 individuals × 2 populations). An rCCR of 1 indicates a single ancestral population. An rCCR of 0.5 is a common heuristic indicating the point of population separation. The relative shape of rCCR curves reflects different separation dynamics such as post-split gene flow[44]. Hash indicates three geographically close population pairs (Mendi–Tari, Bundi–Kundiawa in PNG and Bathurst–Melville in Tiwi) that show recent or incomplete separation. d, rCCRs for population pairs within Australia (with Tiwi samples combined), showing mean (line) and s.e.m. (shading) for 10 replicates. Lower box plots show the estimated times of population separation (rCCR = 0.5). Asterisks indicates a significant difference between the Tiwi–Titjikala and all but one of the other separation times. e, rCCRs for population pairs between Australia and PNG (with PNG samples combined) showing mean (line) and s.e.m. (shading) for 10 replicates. Lower box plots show the estimated times of population separation (rCCR = 0.5) and of the onset of population structure (rCCR = 0.9). Asterisks indicate significant differences. All box plots display the median rCCR across 10 replicates (line), upper and lower quartiles (box), 1.5× interquartile range (whiskers) and outliers (points).

The relatively small recent effective population sizes estimated across Australia were preceded by dramatically larger values 15–20 ka (MSMC2; Fig. 5b and Supplementary Note 6). After a common bottleneck 50–60 ka, as seen in all populations outside Africa1,19, the Australian and Papuan populations grow until about 20 ka, resulting in markedly larger values for all four Australian populations (particularly Tiwi) than seen in PNG. They then decline, coincident with the end of the Last Glacial Maximum. These values are broadly consistent with those obtained using the ABC modelling above.

Population isolation

We use MSMC2 (refs. 42,44) (Supplementary Note 6) to explore the timing and dynamics of population separation via the relative cross coalescence rate (rCCR). Between-population rCCR curves show three distinct clusters (Fig. 5c) indicating that the ancestral Australian and Papuan populations were genetically isolated by 27–30 ka, at least 10,000 years earlier than the establishment of population structure within Australia, which in turn is 5,000–10,000 years earlier than the separation of the ancestral Highland Papuan populations: values consistent with the ABC analysis. The shape and midpoints of the rCCR curves reveal interesting heterogeneity. In Australia, the oldest separation observed is between Tiwi and Titjikala (19 ka), significantly earlier than the separation of other population pairs (Fig. 5d).

We also observe a complex and heterogeneous pattern of isolation between the ancestral Australian and Highland Papuan populations (Fig. 5e). Considering a rCCR value of 0.9 as a proxy for the initial onset of population structure, Titjikala begins isolation from PNG more than 4,000 years earlier than Tiwi or Galiwin’ku, consistent with the above modelling. This pattern then inverts, with the two northern populations becoming fully isolated from PNG more than 2,000 years earlier than Titjikala.

These non-uniform and non-overlapping isolations within Australian and between Australia and PNG show that the establishment of population structure was complex. A likely scenario, consistent with patterns of shared genetic drift (Fig. 3) and demographic modelling (Fig. 4) is that the ancestral populations of both Tiwi and Galiwin’ku remained in genetic contact with the PNG population for a significant period after they had begun to undergo isolation from the Yarrabah–Titjikala population.

Mitochondrial diversity

Until recently, it was thought that no mitochondrial lineages coalesce between Australians and Papuans more recently than 40–50 ka (refs. 7,45), supposedly reflecting the abrupt divergence of ancestral groups after reaching Sahul (the palaeocontinent that includes Australia and New Guinea). The only exceptions were P3b lineages in individuals with Torres Strait Islander ancestry37,45 and a single Q2 lineage from the Kimberley[46]. Recently, two studies incorporating a large collection of mitogenomes of individuals from Oceania reported several other shared Australasian lineages that coalesce more recently than 35–40 ka (refs. 47,48). Supporting this, we observe two lineages with appreciable frequency in Australia (P3 and N13) and divergence times from PNG more recent than 32 ka (Extended Data Fig. 10 and Supplementary Note 7). Using established haplogroup frequencies, we note that these lineages are more frequent in northern Australia (Extended Data Fig. 11), supporting the inferences of non-uniform allele sharing between Australian and Papuan groups from F statistics and the rCCR in the autosomal analyses above.

Discussion

The establishment of this genomic collection has involved more than a decade of consultation with Indigenous leaders, recurring engagement with communities and participants to build mutual trust and a common dialogue and placing the data under Indigenous governance and custodianship. The result is a sizeable cohort with substantial Indigenous ancestry across north and central Australia from people from two independent language families. Comparable studies outside Australia have highlighted the rich genetic diversity in Africa; the bottleneck experienced by all populations outside Africa; the early establishment of population structure across Eurasia; a complex pattern of isolation, migration and extinct hominin ancestry; and the recent considerable expansion of several, but not all, populations[49]. These broad demographic patterns underpin recent advances in our understanding of the genetic basis of common diseases and have enabled the development of tools to aid the diagnosis of rare diseases. However, these may not necessarily relate to or be effective for Indigenous Australians50,51.

We have shown that Indigenous Australians have strong structure relative to other populations outside Africa. By including populations from northern Australia, we have identified a more complex genetic relationship between Indigenous Australians and Papuans than previously inferred[1]. We found that the Tiwi, the only non-Pama–Nyungan language speakers considered here, developed genetic structure from the ancestors of the other Australian communities well before rising sea levels caused the physical separation of the Tiwi islands. Furthermore, non-uniform patterns of shared genetic drift show that this early period was characterized not by discrete separation but rather by an extended period of continuing interaction between the northern populations of Australia and PNG. This was followed by long-term genetic isolation, little detectable migration and strong fluctuation in effective population size, from very large at the end of the Last Glacial Maximum to small and stable over the past few thousand years.

This history has shaped genomic variation in Australia. The early separation of Australians from Eurasians, followed by large effective population sizes of the ancestral Australian populations, have led to the highest levels of previously undescribed private variation observed outside Africa. Notably, 25% of variants are not present in gnomAD, a database approaching saturation for some classes of variation[52]. We observe a depletion of individual heterozygosity and locally common extended haplotypes generating very high levels of ROH and long segments of IBD between individuals. Strong population structure and extended periods of small but stable effective population size almost certainly underpin these observations, rather than recent consanguinity, as observed in more recent population isolates. Failure to account for these signals may confound genomic analyses such as phasing, imputation and association studies, supporting the inclusion of Indigenous Australians in variant databases and resources including genome assemblies.

In addition to population-level applications, our findings are important for individual genomics, including clinical diagnostics. Here, the elevated homozygosity of apparently novel variants specific to Indigenous Australians may falsely lead to them being prioritized as potentially pathogenic. This has implications for any analyses that make judgements about variation in the absence of established phenotypic manifestations, including preconception carrier screening, prenatal diagnostic testing, newborn screening and the prediction of disease predisposition in asymptomatic people. In practice, this points to the need to include individuals from a diverse range of language families and regions.

The value of population-specific reference resources for clinical research and the benefits of personalized medicine have been demonstrated for European populations53,54,55, which are considerably less strongly structured than the communities analysed here. The NCIG collection includes a small fraction of the linguistic, cultural and likely genetic diversity present across Australia. Our results show that no single genomic resource, based on either this collection or current global samples, can adequately capture the genetic diversity present in Indigenous Australians. Importantly, only a relatively small number of individuals from a much wider breadth of communities will be required to overcome this imbalance in the availability of adequate reference data. Ultimately, the engagement, leadership and self-determination of Indigenous people in and through such genomic data will support transformative insights, empowerment, inclusion and equity.

Methods

Inclusion and ethics

The DNA samples analysed in this project form part of a collection of biospecimens, including historically collected samples, maintained under Indigenous governance by the NCIG[11] at the John Curtin School of Medical Research at the Australian National University (ANU). NCIG, a statutory body within ANU, was founded in 2013 and is bound by the National Centre for Indigenous Genomics Statute (2016, updated 2021). This federal government statute requires a majority of Aboriginal and Torres Strait Islander representatives on the NCIG Board, ensuring Indigenous oversight of the centre’s decision-making processes and activities. The board is the custodian of the NCIG collection.

For this project and future work, culturally appropriate community engagement was undertaken[56]. NCIG engaged with traditional owners, community elders and other community representatives to inform the community about the research. This involved contact with the Shire service manager(s), inquiries with community stakeholders, arranging interpreters, promoting the visit in advance and preparing outreach material, including plain-language project summaries and consent forms.

Initial work focused on informing communities about the existence of the historical collection and seeking advice about its continued maintenance and possible future use. During this process, NCIG sought and received with consent (see below) new samples of blood or saliva from current members of the communities we engaged with (some of whom were part of the historical collection). These new samples form the basis of the dataset analysed herein.

Confidentiality agreements, project information and consent forms were communicated to local community organizations, community leaders and participants by means of a community liaison officer, official translation services, local community translators and a video animation. All individuals provided informed personal consent during community visits between circa 2015 and 2018.

The results contained in this paper were returned to communities and all participants using a plain-language summary of the final draft of this manuscript and workshops (two pending) in communities. The community liaison officer was, and is, available to take questions from all participants and community members. The draft of this Article was also available to those who wanted it.

This work was carried out under ANU ethics protocol 2015/065 and the University of Melbourne Ethics protocol 1852770. Further details are in Supplementary Note 1.

Sequencing, read mapping and variant calling

Individuals in the cohort provided a sample of blood or saliva from which DNA was extracted. Genomic DNA quantification, library preparation and sequencing were performed by the Kinghorn Centre for Clinical Genomics (Sydney, Australia). Sequencing was carried out on an Illumina HiSeqX with 150 bp paired end reads to a minimum read depth of 30×. Fastq files were obtained with permission for 60 Papuan samples1,5.

Read mapping and variant calling was carried out as detailed in Supplementary Note 2 to generate the NCIG + PNG autosomal dataset. As needed, this dataset was combined with the low-coverage 1000 Genomes dataset[14] and/or the Simons Genome Diversity Panel (SGDP)[6], subsets of the International Genome Sample Resource (IGSR) collection; SNP array data from Papuan populations[57]; and the high coverage (HC) 1000 Genomes dataset[18]  (see Supplementary Note 2).

High-molecular-weight DNA was extracted from five blood samples, sequenced with Chromium 10x at the KCCG and processed with the Long Ranger WGS software package to generate single-sample phased variant call format files that were used to assess phasing accuracy.

Haplotype inference

Phasing was performed with ShapeIT (v.2.12, default parameters)[58] using both the low-coverage 1000 Genomes reference panel and phase informative reads[59]. Linked-read data were used to estimate switch error rates[60] and select an optimal phasing strategy (Supplementary Note 2).

Ancestry inference

Global ancestry proportions were estimated in the NCIG + PNG dataset using ADMIXTURE (v.1.3)[32] after intersecting with the low-coverage 1000 Genomes dataset and thinning for linkage disequilibrium. K was varied from 2 to 12 in cross-validation mode with ancestry proportions inferred at K = 6 and verified via principal component analysis[61], F_4 ratios[36] and RFMIX[62] (Supplementary Note 2).

Local ancestry was inferred using RFMIX (v.1.5.4) with a reference panel of individuals from the NCIG + PNG dataset inferred to have mainly Indigenous ancestry (Supplementary Note 2) and European, East Asian, South Asian and African individuals from the low-coverage 1000 Genomes dataset (see Supplementary Note 2 for parameters and composition of the reference panel). Genomic coordinates were identified for each individual that demarcate regions where one or both haplotypes were of neither Indigenous Australian nor Papuan ancestry, generating a ‘mask’ coordinate file in BED format and a VCF file with variant calls in these regions set to missing. The mask was used to keep all regions of the genome for which both haplotypes have Indigenous Australian or Papuan ancestry and remove all other regions. We refer to this dataset as NCIG + PNG (masked). This masking pipeline was validated using F_4 ratios, ADMIXTURE and principal component analysis, run with the ‘lsqproject’ feature of the EIGENSTRAT software package (EIGENSOFT v.7.2.1)[61]. This mask removed more than 95% of the genome for five individuals who were not considered in subsequent analysis.

Kinship inference

A subset of 150 unrelated individuals (97 Australian and 53 PNG), up to second-degree relatives (that is, no second-degree relatives or closer present), were identified using KING[63] with the ‘--unrelated’ and ‘--degree 2’ options from the NCIG + PNG dataset (without ancestry masking). Downstream analyses of population structure revealed eight Tiwi samples from this subset of 150 to cluster in a pattern consistent with one or more of their ancestors being of non-Tiwi Indigenous ancestry (designated ‘Tiwi outliers’; an additional two ‘Tiwi outliers’ were removed with the relatedness filter (Supplementary Note 2)). Unless otherwise stated, all main analyses were performed on this ancestry-masked, unrelated and non-outlier subsample, which included 142 samples: 89 from the NCIG collection (34 Tiwi, 31 Yarrabah, 17 Galiwin’ku, 7 Titjikala) and 53 from PNG (25 Highland PNG, 28 Island PNG). For comparison, ref. [1] analyses 69 Australian samples with similar constraints.

Genomic variation

To assess variant sharing, the NCIG + PNG (masked) dataset was merged with the high-coverage 1000 Genomes dataset[18] (both underwent equivalent data processing, including variant quality score recalibration filtering at 99.8), taking the union of sites using the PLINK ‘--bmerge’ command[64] and removing sites that became triallelic using the ‘--exclude’ command.

Variants were assigned to one of four non-overlapping categories as defined previously[14]; observed in a single-population sample (‘population private’); observed in more than one population sample within a single continent (‘continent private’); observed in several, but not all, continents (‘shared across some continents’); and observed in all continents (‘shared across all continents’).

To allow an unbiased comparison, each population sample was restricted to five unrelated individuals using the PLINK ‘--keep’ command (Yarrabah and Island Melanesia (PNG (Is.)) were restricted to the five least-admixed unrelated individuals). Given the potential of relatedness to reduce the levels of variation in these subsamples, we confirmed that no pairs of individuals within Galiwin’ku, Tiwi, Titjikala and PNG (HL) had detectable relatedness up to the fourth degree (the maximum threshold identified by the KING algorithm). The difficulty of obtaining a subset of both unrelated and unadmixed samples from Yarrabah and PNG (Is.) necessitated the inclusion of two pairs of third-degree relatives from Yarrabah.

Allele frequency reports stratified by population and continent were generated using the PLINK ‘--freq’ command (Fig. 1a,b). This analysis, with equal sample size of n = 5, is shown for all populations of the 1000 Genomes dataset in Supplementary Fig. 1a and was repeated on the full dataset (that is, without subsampling individuals) both with ancestry masking (Supplementary Fig. 1b) and without (Supplementary Fig. 1c) and on versions of the masked dataset filtered to a sample size of n = 15 and n = 25 unrelated samples per population (Supplementary Fig. 1d,e).

The above analysis was repeated after subsetting to only sites classified as ‘pathogenic’, ‘likely pathogenic’ or ‘drug response’ in ClinVar (release 20230514; Supplementary Fig. 1f) and after subsetting to non-synonymous variants within the type 2 diabetes associated genes listed in Tables 2 and 3 of ref. [23] (Supplementary Note 3). Coordinates of these genes were obtained from GENCODE Release 37 (GRCh38.p13), and non-synonymous variants within the NCIG + PNG + 1000 G (high-coverage) dataset were identified using VEP[65].

Minor alleles were defined using the PLINK ‘--recode’ command in the above dataset (restricted to five individuals per population sample), where the minor allele is defined in reference to the whole dataset. The allele count within a population sample was recorded using the PLINK ‘--freq’ command and binned from count 1 (seen once in a set of 10 haplotypes) to 10 (fixed in the sample) to generated allele frequency plots (Fig. 1c).

Per-individual counts of heterozygous sites were produced from the full dataset after ancestry masking (NCIG + PNG (masked) + high-coverage 1000 Genomes), with values rescaled to account for the proportion of the genome ancestry masked in each sample (open circles in Fig. 1d). For individuals with more than 5% ancestry other than Indigenous ancestry, these values were also generated from the unmasked dataset (NCIG + PNG + high-coverage 1000 Genomes) (dashes in Fig. 1d).

Phenotypic impact was predicted for amino acid substitutions in the full dataset (both unmasked and masked) using the VEP ‘--sift b –polyphen b –custom ClinVar_20200210/clinvar.vcf.gz,ClinVar,vcf,exact,0,CLNSIG,CLNREVSTAT,CLNDN –coding_only’ command. Amino acid substitutions with a SIFT score less than 0.05 were considered potentially functional[24], and the number of such homozygous non-reference sites was counted per individual. Unmasked and rescaled values are shown as defined above (Fig. 1e). ‘Pathogenic’ ClinVar annotations were also counted (Supplementary Table 1).

Runs of homozygosity

The number of ROH segments greater than 1 megabase (Mb) and the sum of their length were estimated using bcftools roh[66] (v.1.11, default parameters) in the NCIG + PNG + high-coverage 1000 Genomes dataset (Fig. 1f and Extended Data Fig. 1b,c) and separately for the SGDP dataset. Given that we are interested in per-individual ROH regardless of recent ancestry, unmasked data were used. Individuals with more than 5% ancestry other than Indigenous ancestry are displayed as dashes in Fig. 1f. For comparison, we show individuals from the SGDP dataset with the most extreme ROH (and their population sample) in Extended Data Fig. 1c.

Segregating sites and progressive sampling

The number of polymorphic sites observed was calculated as the per-population sample size was progressively increased using the NCIG + PNG (masked) + high-coverage 1000 Genomes dataset. Yarrabah and PNG (Is.) were not included because of variable ancestry other than Indigenous ancestry, and only unrelated individuals with less than 5% ancestry masked were included for the other populations. The count of segregating sites was obtained using the PLINK ‘--freq’ command and custom Unix scripts as the sample size was progressively increased from 1 to 35, taking the average of ten replicates (Fig. 1g).

The level of novel variation observed in a continent, given that all other continents have already been sampled, was estimated for the same dataset with the reintroduction of unrelated individuals from Yarrabah and PNG (Is.) with less than 25% ancestry masked (four individuals from Yarrabah and two from PNG (Is.)). This less-stringent cutoff ensured that a similar number of populations were included from each continent. Populations were pooled into continental groups, and the number of further polymorphic sites observed was scored as the sample was progressively increased from 1 to 80, after first sampling 80 individuals from each of the other five continents, taking the average of ten replicates (Fig. 1g).

Population structure

Pairwise genetic distances were estimated using the minor allele frequency-corrected covariance (COV)33,61 (Extended Data Fig. 2a) calculated using PLINK (v.1.9)[64]; rare allele sharing (Fig. 2d), defined by allele count less than or equal to 5 in the NCIG + PNG (masked, all individuals) dataset; and pairwise outgroup F_3 scores using ADMIXTOOLS (v.5.1, default settings)[36] (Extended Data Fig. 2b). Ancestry was masked and analysis restricted to sites without missing data in each pairwise comparison; full details are in Supplementary Note 4.

Hierarchical clustering was carried out using the hclust() function of the stats package of R[67] on the pairwise outgroup F_3 matrix, with relatedness filtering (Fig. 2b).

The ADMIXTURE algorithm[32] was applied to the NCIG + PNG (masked) dataset with all samples (Extended Data Fig. 3) and after relatedness filtering (Fig. 2c). K was varied from 2 to 8, with cross validation supporting K = 4 and K = 5 (Supplementary Note 4).

The RefinedIBD algorithm (v.102)[68] was used to infer IBD tract sharing between pairs of individuals in the NCIG + PNG (masked) dataset (Fig. 2d). Variants with a minor allele count of strictly fewer than 8 in the dataset were removed. Default settings were used, including a threshold of 1.5 cM as the minimum IBD segment length. Counts were rescaled to account for the proportion of the genome missing because of masking in each pairwise comparison.

Multidimensional scaling (MDS) was applied to the COV matrix using the cmdscale() function in R (v.5.1) following the approach of ref. [69] (Extended Data Fig. 2c).

UMAP (v.0.2.7.0)[70] was applied as per ref. [34] to the top ten components of the MDS output generated from the COV matrix (Fig. 2e).

fineSTRUCTURE (v.4.0.1)31,33 was run on unrelated individuals with no discernible ancestry other than Indigenous ancestry from the NCIG + PNG (unmasked) dataset (no individuals from Yarrabah were included because of the requirement for no missing data; Fig. 2f and Extended Data Fig. 4; see Supplementary Note 4 for full details).

To contextualize levels of structure observed among Indigenous Oceanic populations, the hierarchical clustering, ADMIXTURE and RefinedIBD algorithms were applied to other continental cohorts from the 1000 Genomes dataset (Supplementary Note 4).

Pairwise FST was calculated for the Australian and PNG population samples and those of SGDP using the NCIG + PNG (masked) + 1000 G (low-coverage) + SG dataset. FST was calculated using the Eigenstrat software package[61]. To provide an unbiased estimator of FST[71], the dataset was filtered to a subset of sites that were polymorphic in the Mbuti populations of the SGDP collection. The results are shown in Extended Data Fig. 7.

F statistics

F statistics were calculated using the NCIG + PNG (masked) + 1000 G (low-coverage) dataset, with further datasets included as described below. ADMIXTOOLS[36] was used to calculate all F statistics, using the Yoruban (YRI) population from the 1000 Genomes as the outgroup, with default parameters, unless otherwise stated.

The degree of shared genetic drift between each Indigenous Australian sample and a panel of Papuan samples was estimated using the statistic F_3(YRI; PNG, NCIGx). Here ‘PNG’ is the panel of 25 Highland PNG samples described in ref. [1] and ‘NCIGx’ represents each Indigenous Australian individual assessed in turn. Significantly higher values of this statistic indicate a population shares more genetic drift with PNG, relative to the other populations (Fig. 3a and Supplementary Note 4).

F_4-statistics of the form F_4(T)(YRI, PNG; X, Y)[72] were used to infer differing degrees of shared genetic drift between pairs of the Australian populations and PNG. Population nomenclature is as described above, with ‘X’ and ‘Y’ representing sets of samples from all pairwise combinations of Tiwi, Galiwin’ku, Yarrabah and Titjikala. As is standard[72], we defined Z-scores greater than absolute value 3 to be significant, meaning Y shares more drift with PNG than X (positive score).

To determine whether populations from South Asia, East Asia or Oceania share the same degree of genetic drift with Titjikala and either Tiwi or Galiwin’ku, F_4-statistics of the form \({F}_{4}^{({\rm{T}})}\) (Asia-Y, YRI; Australia-X, Titjikala) were calculated on an expanded dataset including the SGDP (Supplementary Note 2), where ‘Asia-Y’ is any SGDP sample from South Asia, East Asia or Oceania; and ‘Australia-X’ is either the Tiwi or Galiwin’ku sample (Fig. 3b; further details and theoretical justification are given in Supplementary Note 4).

F_3-statistics of the form F_3(AUAx; PNG, AUAy) were used to assess whether the increased affinity the three northern populations of Australia (Tiwi, Galiwin’ku and Yarrabah) hold with PNG can be attributed to recent Papuan-related admixture. Here ‘PNG’ represents the 25 Highland Papuans, and ‘AUAx’ and ‘AUAy’ represent one of Tiwi, Galiwin’ku, Titjikala and Yarrabah. There is significant evidence that the population ‘AUAx’ has recently received an ancestral contribution from a population related to ‘PNG’ and ‘AUAy’ if the statistic is less than −3 (Extended Data Fig. 8c and Supplementary Note 4).

To test whether the additional genetic drift shared between Papuan populations and Tiwi (relative to Titjikala) was uniform across Papuan groups, we incorporated single-nucleotide polymorphism array data from PNG[57] and compared the outgroup F_3 statistics F_3(YRI; Tiwi, PNG-X) to F_3(YRI; Titjikala, PNG-X) (Supplementary Notes 2 and 4).

Demographic modelling of the historical relationships within Australia

We use ABC to assess a range of demographic topologies. Seven plausible topologies were identified and datasets simulated 50,000 times from each with msprime (v.1 within tskit release)38,73. The following summary statistics were calculated: F_3 and F_4 statistics, the second and third moments of each F_3 and F_4 statistic, Tajima’s D, nucleotide diversity and counts of segregating sites. Statistics were computed directly from tree sequences using the tskit package (development version, since released as v.1.0)[74]. The same set of summary statistics were computed on the NCIG + PNG dataset using ADMIXTOOLS[36] and PLINK[64]. We checked that the statistics were calculated the same way and return the same values using all software. An ABC–random forest model[75] was used to infer the most probable scenario and estimate model parameters (Supplementary Note 5).

Historic autosomal effective population size and isolation

Pairwise IBD tracts were inferred using RefinedIBD (v.102)[76], and recent effective population sizes were inferred using IBDNe (v.23Apr20.ae9)[41], with ancestry-specific effective population sizes (ref. [77]) inferred for Yarrabah and PNG (Is.) using the local ancestry inferred from RFMIX (parameters and sample sizes are detailed in Supplementary Note 6).

Longer-term effective population sizes were inferred with MSMC2 (v.2.1.2)1,42 from eight phased haplotypes from four randomly sampled individuals from each population (all autosomes), repeated for five replicates of unique sets of four individuals (some individuals may appear in more than one replicate) and applying masks for mappability, low coverage and ancestry other than Indigenous ancestry (Supplementary Note 6).

Genetic isolation between population pairs was inferred with MSMC2 rCCR using ten replicates of four phased haplotypes per population (two individuals).

Mitochondrial genetic structure and diversity

Mitochondrial variants were called with GATK (v.3.8-0)[78] ‘HaplotypeCaller’ with ploidy set to haploid and validated via several metrics including maternal parent–offspring genotype concordance (Supplementary Note 7). Mitochondrial phylogenies were inferred using BEAST (v.2.6.0)[79], and maximum clade credibility trees were produced with TreeAnnotator[79]. Further Australian and Melanesian mitochondrial sequences were incorporated to better resolve the points of coalescence between lineages (Supplementary Note 7). A dataset of mitochondrial haplogroup frequencies from previous studies was collated to explore the frequencies of haplogroups N13, Q2 and P3 across Australia (Supplementary Note 7).

Maps

Maps were obtained from Google Maps using the ‘get_googlemap’ function of the ‘ggmap’ package in R[80], and points were superimposed using ggplot2 (ref. [81]).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

All sequencing data, variant calls and metadata have been deposited in the Australian National Computational Infrastructure, Canberra, under project identifier TE53. Access can be requested by writing to the NCIG Collection Access and Research Advisory Committee, overseen by the Indigenous-majority NCIG Board, at jcsmr.ncig@anu.edu.au. The data are available for general research use subject to meeting the requirements of the NCIG Governance Framework available at https://ncig.anu.edu.au/files/NCIG-Governance-Framework.pdf. Requests for data access for external research will be assessed in accordance with the NCIG Governance Framework.

References

1. Malaspinas, A. S. et al. A genomic history of Aboriginal Australia. Nature 538, 207–214 (2016).

2. Henn, B. M. et al. Distance from sub-Saharan Africa predicts mutational load in diverse human genomes. Proc. Natl Acad. Sci. USA 113, E440–E449 (2016).

3. Rasmussen, M. et al. An Aboriginal Australian genome reveals separate human dispersals into Asia. Science 334, 94–98 (2011).

4. Jacobs, G. S. et al. Multiple deeply divergent denisovan ancestries in Papuans. Cell 177, 1010–1021.e32 (2019).

5. Vernot, B. et al. Excavating Neandertal and Denisovan DNA from the genomes of Melanesian individuals. Science 352, 235–239 (2016).

6. Mallick, S. et al. The Simons Genome Diversity Project: 300 genomes from 142 diverse populations. Nature 538, 201–206 (2016).

7. Tobler, R. et al. Aboriginal mitogenomes reveal 50,000 years of regionalism in Australia. Nature 544, 180–184 (2017).

8. Bouckaert, R. R., Bowern, C. & Atkinson, Q. D. The origin and expansion of Pama–Nyungan languages across Australia. Nat. Ecol. Evol. 2, 741–749 (2018).

9. McConvell, P. & Bowern, C. The prehistory and internal relationships of Australian languages. Lang. Linguist. Compass 5, 19–32 (2011).

10. Barbieri, C. et al. A global analysis of matches and mismatches between human genetic and linguistic histories. Proc. Natl Acad. Sci. USA 119, e2122084119 (2022).

11. Australian National University. National Centre for Indigenous Genomics Statute (2021); www.legislation.gov.au/Details/F2021L00183.

12. Peterson, N. & Taylor, J. Demographic transition in a hunter-gatherer population: the Tiwi case, 1929–1996. Aust. Aborig. Stud. 1, 11–27 (1998).

13. Tindale, N. Genealogical Data on the Aborigines of Australia, Vol. 2 (1938–1939) (Department of Aboriginal and Torres Strait Islander Partnerships, Community and Personal Histories Removals Database; originally held by the Museum of South Australia, 1938).

14. The 1000 Genomes Project Consortium. A global reference for human genetic variation. Nature 526, 68–74 (2015).

15. Nagle, N. et al. Antiquity and diversity of aboriginal Australian Y-chromosomes. Am. J. Phys. Anthropol. 159, 367–381 (2016).

16. McEvoy, B. P. et al. Whole-genome genetic diversity in a sample of Australians with deep Aboriginal ancestry. Am. J. Hum. Genet. 87, 297–305 (2010).

17. Bergström, A. et al. Deep roots for Aboriginal Australian Y chromosomes. Curr. Biol. 26, 809–813 (2016).

18. Byrska-Bishop, M. et al. High-coverage whole-genome sequencing of the expanded 1000 Genomes Project cohort including 602 trios. Cell 185, 3426–3440.e19 (2022).

19. Bergström, A. et al. Insights into human genetic variation and population history from 929 diverse genomes. Science 367, eaay5012 (2020).

20. Karczewski, K. J. et al. The mutational constraint spectrum quantified from variation in 141,456 humans. Nature 581, 434–443 (2020).

21. Henn, B. M., Cavalli-Sforza, L. L. & Feldman, M. W. The great human expansion. Proc. Natl Acad. Sci. USA 109, 17758–17764 (2012).

22. Friedlaender, J. S. et al. The genetic structure of Pacific Islanders. PLoS Genet. 4, e19 (2008).

23. Xue, A. et al. Genome-wide association analyses identify 143 risk variants and putative regulatory mechanisms for type 2 diabetes. Nat. Commun. 9, 2941 (2018).

24. Ng, P. C. & Henikoff, S. Predicting deleterious amino acid substitutions. Genome Res. 11, 863–874 (2001).

25. Adzhubei, I. A. et al. A method and server for predicting damaging missense mutations. Nat. Methods 7, 248–249 (2010).

26. Landrum, M. J. & Kattman, B. L. ClinVar at five years: delivering on the promise. Hum. Mutat. 39, 1623–1630 (2018).

27. Kirin, M. et al. Genomic runs of homozygosity record population history and consanguinity. PLoS ONE 5, e13996 (2010).

28. Hermes, A. et al. Beyond platitudes: a qualitative study of Australian Aboriginal people’s perspectives on biobanking. Intern Med. J. 51, 1426–1432 (2021).

29. International HapMap 3 Consortium. Integrating common and rare genetic variation in diverse human populations. Nature 467, 52–58 (2010).

30. Marchini, J., Cardon, L. R., Phillips, M. S. & Donnelly, P. The effects of human population structure on large genetic association studies. Nat. Genet. 36, 512–517 (2004).

31. Leslie, S. et al. The fine-scale genetic structure of the British population. Nature 519, 309–314 (2015).

32. Alexander, D. H., Novembre, J. & Lange, K. Fast model-based estimation of ancestry in unrelated individuals. Genome Res. 19, 1655–1664 (2009).

33. Lawson, D. J., Hellenthal, G., Myers, S. & Falush, D. Inference of population structure using dense haplotype data. PLoS Genet. 8, e1002453 (2012).

34. Diaz-Papkovich, A., Anderson-Trocmé, L., Ben-Eghan, C. & Gravel, S. UMAP reveals cryptic population structure and phenotype heterogeneity in large genomic cohorts. PLoS Genet. 15, e1008432 (2019).

35. Reich, D., Thangaraj, K., Patterson, N., Price, A. L. & Singh, L. Reconstructing Indian population history. Nature 461, 489–494 (2009).

36. Patterson, N. et al. Ancient admixture in human history. Genetics 192, 1065–1093 (2012).

37. Nagle, N. et al. Mitochondrial DNA diversity of present-day Aboriginal Australians and implications for human evolution in Oceania. J. Hum. Genet. 62, 343–353 (2017).

38. Baumdicker, F. et al. Efficient ancestry and mutation simulation with msprime 1.0. Genetics 220, iyab229 (2022).

39. Raynal, L. et al. ABC random forests for Bayesian parameter inference. Bioinformatics 35, 1720–1728 (2019).

40. Nielsen, S. V. et al. Bayesian inference of admixture graphs on Native American and Arctic populations. PLoS Genet. 19, e1010410 (2023).

41. Browning, S. R. & Browning, B. L. Accurate non-parametric estimation of recent effective population size from segments of identity by descent. Am. J. Hum. Genet. 97, 404–418 (2015).

42. Schiffels, S. & Wang, K. MSMC and MSMC2: the multiple sequentially Markovian coalescent. Methods Mol. Biol. 2090, 147–166 (2020).

43. Yunusbaev, U. et al. Reconstructing recent population history while mapping rare variants using haplotypes. Sci. Rep. 9, 5849 (2019).

44. Schiffels, S. & Durbin, R. Inferring human population size and separation history from multiple genome sequences. Nat. Genet. 46, 919–925 (2014).

45. Nagle, N. et al. Aboriginal Australian mitochondrial genome variation – an increased understanding of population antiquity and diversity. Sci. Rep. 7, 43041 (2017).

46. Hudjashov, G. et al. Revealing the prehistoric settlement of Australia by Y chromosome and mtDNA analysis. Proc. Natl Acad. Sci. USA 104, 8726–8730 (2007).

47. Pedro, N. et al. Papuan mitochondrial genomes and the settlement of Sahul. J. Hum. Genet. 65, 875–887 (2020).

48. Purnomo, G. A. et al. Mitogenomes reveal two major influxes of Papuan ancestry across Wallacea following the last glacial maximum and Austronesian contact. Genes (Basel) 12, 965 (2021).

49. Nielsen, R. et al. Tracing the peopling of the world through genomics. Nature 541, 302–310 (2017).

50. Easteal, S. et al. Equitable expanded carrier screening needs indigenous clinical and population genomic data. Am. J. Hum. Genet. 107, 175–182 (2020).

51. Baynam, G. et al. A germline MTOR mutation in Aboriginal Australian siblings with intellectual disability, dysmorphism, macrocephaly, and small thoraces. Am. J. Med. Genet. A 167, 1659–1667 (2015).

52. Chen, S. et al. A genome-wide mutational constraint map quantified from variation in 76,156 human genomes. Preprint at bioRxiv https://doi.org/10.1101/2022.03.20.485034 (2022).

53. Deelen, P. et al. Improved imputation quality of low-frequency and rare variants in European samples using the ‘Genome of The Netherlands’. Eur. J. Hum. Genet. 22, 1321–1326 (2014).

54. Gudbjartsson, D. F. et al. Large-scale whole-genome sequencing of the Icelandic population. Nat. Genet. 47, 435–444 (2015).

55. Mitt, M. et al. Improved imputation accuracy of rare and low-frequency variants using population-specific high-coverage WGS-based imputation reference panel. Eur. J. Hum. Genet. 25, 869–876 (2017).

56. Huebner, S., Hermes, A. & Easteal, S. in Indigenous Research Ethics: Claiming Research Sovereignty Beyond Deficit and the Colonial Legacy, Vol. 6 (eds George, L., Tauri, J. & MacDonald, L. T. A. o T.) Ch. 8 (Emerald, 2020).

57. Bergström, A. et al. A Neolithic expansion, but strong genetic structure, in the independent history of New Guinea. Science 357, 1160–1163 (2017).

58. Delaneau, O., Marchini, J. & Zagury, J. F. A linear complexity phasing method for thousands of genomes. Nat. Methods 9, 179–181 (2012).

59. Delaneau, O., Howie, B., Cox, A. J., Zagury, J. F. & Marchini, J. Haplotype estimation using sequencing reads. Am. J. Hum. Genet. 93, 687–696 (2013).

60. Choi, Y., Chan, A. P., Kirkness, E., Telenti, A. & Schork, N. J. Comparison of phasing strategies for whole human genomes. PLoS Genet. 14, e1007308 (2018).

61. Patterson, N., Price, A. L. & Reich, D. Population structure and eigenanalysis. PLoS Genet. 2, 2074–2093 (2006).

62. Maples, B. K., Gravel, S., Kenny, E. E. & Bustamante, C. D. RFMix: a discriminative modeling approach for rapid and robust local-ancestry inference. Am. J. Hum. Genet. 93, 278–288 (2013).

63. Manichaikul, A. et al. Robust relationship inference in genome-wide association studies. Bioinformatics 26, 2867–2873 (2010).

64. Purcell, S. et al. PLINK: a tool set for whole-genome association and population-based linkage analyses. Am. J. Hum. Genet. 81, 559–575 (2007).

65. McLaren, W. et al. The Ensembl variant effect predictor. Genome Biol. 17, 122 (2016).

66. Danecek, P. et al. Twelve years of SAMtools and BCFtools. Gigascience 10, giab008 (2021).

67. R Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2009); https://www.R-project.org/.

68. Browning, B. L. & Browning, S. R. Improving the accuracy and efficiency of identity-by-descent detection in population data. Genetics 194, 459–471 (2013).

69. Browning, S. R. et al. Local ancestry inference in a large US-based Hispanic/Latino study: Hispanic Community Health Study/Study of Latinos (HCHS/SOL). G3 6, 1525–1534 (2016).

70. McInnes, L., Healy, J., Saul, N. & Großberger, L. UMAP: Uniform Manifold Approximation and Projection. J. Open Source Softw. 3, 861 (2018).

71. Bhatia, G., Patterson, N., Sankararaman, S. & Price, A. L. Estimating and interpreting FST: the impact of rare variants. Genome Res. 23, 1514–1521 (2013).

72. Peter, B. M. Admixture, population structure, and f-statistics. Genetics 202, 1485–1501 (2016).

73. Kelleher, J., Etheridge, A. M. & McVean, G. Efficient coalescent simulation and genealogical analysis for large sample sizes. PLoS Comput. Biol. 12, e1004842 (2016).

74. Ralph, P., Thornton, K. & Kelleher, J. Efficiently summarizing relationships in large samples: a general duality between statistics of genealogies and genomes. Genetics 215, 779–797 (2020).

75. Pudlo, P. et al. Reliable ABC model choice via random forests. Bioinformatics 32, 859–866 (2016).

76. Browning, B. L. & Browning, S. R. Detecting identity by descent and estimating genotype error rates in sequence data. Am. J. Hum. Genet. 93, 840–851 (2013).

77. Browning, S. R. et al. Ancestry-specific recent effective population size in the Americas. PLoS Genet. 14, e1007385 (2018).

78. McKenna, A. et al. The Genome Analysis Toolkit: a MapReduce framework for analyzing next-generation DNA sequencing data. Genome Res. 20, 1297–1303 (2010).

79. Drummond, A. J. & Rambaut, A. BEAST: Bayesian evolutionary analysis by sampling trees. BMC Evol. Biol. 7, 214 (2007).

80. Kahle, D. & Wickham, H. ggmap: spatial visualization with ggplot2. R J. 5, 144–161 (2013).

81. Wickham, H. ggplot2: Elegant Graphics for Data Analysis (Springer-Verlag, 2016).

Acknowledgements

We acknowledge the Aboriginal and Torres Strait Islander peoples as the first peoples and traditional custodians of the lands and waters where we meet, live, learn and work. We celebrate the rich diversity of Aboriginal and Torres Strait Islander cultures and the continuing leadership of our First Nations’ peoples and communities who have paved the way. We pay our respects to ancestors of this country, the legacy of elders, the knowledge holders and leaders of the past, present and future. We are indebted to the individuals and communities who participated in this research and the NCIG Governance Board who guided this work in a culturally appropriate manner. We acknowledge the following community organizations and individuals: Yarrabah Shire Council, R. Andrews, E. Fourmile and P. Burns; Tiwi Land Council Board members; Yalu Aboriginal Corporation (Galiwin’ku), R. Wunungmurra, E. Djotja, R. Gundjarrangbuy; Titjikala Shire Council and Titjikala Health Services. We thank A. Brown, G. Mann, M. Dinger and B. Llamas for helpful discussion and feedback. We thank J. McCluskey and K. Nugent for guidance and support. This work was supported in part by NHMRC grants nos. APP1143734, APP2021644, APP2021172 and APP2011277, Bioplatforms Australia and the National Computational Merit Allocation Scheme. This work was conducted on land traditionally owned by the Ngunnawal and Ngambri peoples and the Wurundjeri people of the Kulin Nation.

Author information

Author notes

1. These authors contributed equally: Matthew Silcocks, Ashley Farlow, Azure Hermes (Gimuy Walubara Yidinji)

2. These authors jointly supervised this work: Simon Easteal, Stephen Leslie

Authors and Affiliations

1. National Centre for Indigenous Genomics, John Curtin School of Medical Research, Australian National University, Canberra, Australian Capital Territory, Australia

Matthew Silcocks, Ashley Farlow, Azure Hermes (Gimuy Walubara Yidinji), Hardip R. Patel, Sharon Huebner, Gareth Baynam, Misty R. Jenkins (Gunditjmara), Simon Easteal & Stephen Leslie

2. University of Melbourne, School of Biosciences, Parkville, Victoria, Australia

Matthew Silcocks, Stephen Leslie & Stephen Leslie

3. University of Melbourne, School of Mathematics and Statistics, Parkville, Victoria, Australia

Ashley Farlow, Georgia Tsambos, Damjan Vukcevic, Stephen Leslie, Ashley Farlow & Stephen Leslie

4. Faculty of Health and Medical Sciences, Division of Paediatrics and Telethon Kids Institute, University of Western Australia, Perth, Western Australia, Australia

Gareth Baynam & Gareth Baynam

5. Western Australian Register of Developmental Anomalies, King Edward Memorial Hospital and Rare Care Centre, Perth Children’s Hospital, Perth, Western Australia, Australia

Gareth Baynam & Gareth Baynam

6. Immunology Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia

Misty R. Jenkins (Gunditjmara) & Misty R. Jenkins (Gunditjmara)

7. University of Melbourne, Department of Medical Biology, Parkville, Victoria, Australia

Misty R. Jenkins (Gunditjmara) & Misty R. Jenkins (Gunditjmara)

8. John Curtin School of Medical Research, Australian National University, Canberra, Australian Capital Territory, Australia

Azure Hermes (Gimuy Walubara Yidinji), Hardip R. Patel, Sharon Huebner & Simon Easteal

Consortia

The National Centre for Indigenous Genomics

- Matthew Silcocks

- , Ashley Farlow

- , Azure Hermes (Gimuy Walubara Yidinji)

- , Hardip R. Patel

- , Sharon Huebner

- , Gareth Baynam

- , Misty R. Jenkins (Gunditjmara)

- , Simon Easteal

- & Stephen Leslie

Contributions

A.F., A.H., D.V., G.B., M.R.J., S.E. and S.L. designed the study. A.H. collected the samples and was responsible for community engagement. A.F., H.R.P. and M.S. prepared the data. A.F., G.T., M.S. and S.L. performed the analyses. A.H., S.E. and S.H. contributed to the manuscript with regard to community consultation, ethics and the NCIG collection. A.F., M.S. and S.L. wrote the manuscript. All authors read and approved the manuscript.

Corresponding author

Correspondence to Stephen Leslie.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature thanks Andres Moreno-Estrada, Nicola Mulder and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Global ancestry and homozygosity.

A. Global ancestry proportions for NCIG, Papuan and 1000 Genomes. The software ADMIXTURE run with 159 Indigenous Australian samples, 60 Papuan and 2,600 samples from the LC 1000 Genomes. Samples shown horizontally by population (with reduced bar width for the 1000 Genomes samples) and cluster (colour) assignment proportion shown as bar height. ADMIXTURE was run assuming the sample contained from k = 2 up to k = 12 clusters (y-axis). No ancestry mask applied. Restricted to biallelic SNVs, MAF > 0.01 and LD thinned. This analysis was used to estimate non-Indigenous Australian or PNG ancestry in the NCIG and PNG samples. B. Runs of homozygosity (ROH) for the NCIG + PNG (unmasked) dataset and a subset of (HC) 1000 Genomes samples. Mean count versus mean sum of ROH segments greater than 1 Mb in length. Error bars are within population SEM. Note that a long-term reduction in effective population size is expected to increase both the count and total length of ROH (as seen for NCIG populations), whereas recent consanguinity generates a small number of long ROH. C. Per-individual ROH length (for ROH > 1 Mb) as a percentage of the total autosomal genome length (2.8 Gb). Individuals from three Indigenous populations from Nepal (Kusunda) and Brazil (Surui and Karitiana) from the SGDP were included for comparison as some exhibited extreme levels of ROH. Individuals identified as “Tiwi-outliers” were included in the Tiwi sample in this plot and are evident in panel C as a cluster of individuals with reduced ROH.

Extended Data Fig. 2 Genetic distances and population structure within Oceania.

A. Heatmap of the minor allele frequency corrected pairwise covariance (COV) values between all individuals in the NCIG + PNG (masked) dataset (including related individuals). Individuals are listed in the same order along each axis and the population they were sampled from is indicated along the axes. Note that higher values of genotype covariance indicate greater genetic similarity. B. Heatmap of pairwise outgroup F_3-statistics of the form F_3(YRI;AUAx,AUAy), where AUAx and AUAy are any pair of individuals from the NCIG + PNG (masked) dataset (including related individuals). Individuals are listed in the same order along each axis and the population they were sampled from is indicated along the axes. Higher values indicate greater genetic similarity. C. Multidimensional scaling applied to the pairwise genotype covariance (COV) matrix estimated from the NCIG + PNG (masked) dataset after filtering to unrelated individuals. The first two dimensions are shown. (see Methods for further details of all three plots).

Extended Data Fig. 3 ADMIXTURE ancestry assignment.

The clustering algorithm ADMIXTURE applied to the NCIG + PNG (masked) dataset (including related individuals) assuming K = 2 to K = 8 clusters (subpopulations) are represented in the data (See Methods). Clustering makes no reference to the sampling locations of the individuals and is based on genetic data alone. Individuals are listed along the x-axis, grouped according to their sampling location, with bars above reflecting their cluster assignment in the following manner: each inferred cluster is labelled by a colour and the proportion of bar assigned that colour represents the probability that the individual is assigned to that cluster. Colours were manually selected (post-hoc) for K = 7 to match the labels in panels 2 A and 2B of Fig. 2, and for other values of K the colouring scheme was merged or split as appropriate. Also shown are the cross-validation (CV) scores used for model selection.

Extended Data Fig. 4 Fine-scale genetic structure within Oceania.

A. Hierarchical clustering tree (up to 13 clusters) produced from the maximum a posteriori state partitions inferred by fineSTRUCTURE (see Methods). The NCIG + PNG (masked) dataset used for this analysis was reduced to a subset of unadmixed and unrelated samples. Samples are coloured according to their sampling location. Note that the 25 samples from Papua New Guinea are denoted by their sub-sampling locations in this analysis (Bundi, Kundiawa (Kuman), Marawaka, Mendi and Tari). B. Clustering of the NCIG + PNG (masked) dataset into 5 clusters based only on genetic data using fineSTRUCTURE. The dataset used for this analysis was reduced to a subset of unadmixed and unrelated samples. For each individual, the coloured symbol represents the genetic cluster to which the individual is assigned. C. The fineSTRUCTURE coincidence matrix showing the proportion of cluster partitions in which two individuals are grouped in the same cluster during the MCMC. The NCIG + PNG (masked) dataset used for this analysis was reduced to a subset of unadmixed and unrelated samples.

Extended Data Fig. 5

Global IBD tract sharing. Heatmaps depicting the number of tracts shared IBD (inferred using the RefinedIBD algorithm; see Methods and Supplementary Note 4) within four continental samples from the 1000 Genomes collection: A. Europe, B. East Asia, C. Africa and D. South Asia. A comparable plot featuring NCIG samples is presented in Fig. 2d of the Main Text. Comparisons of samples inferred to share a familial relationship (2nd degree or closer) were masked in red. Note that a different scale was used for each heatmap to maximise definition.

Extended Data Fig. 6

Global population structure. Results of the ADMIXTURE algorithm and hierarchical clustering of outgroup F_3-statistic values for four continental samples from the 1000 Genomes collection; A. Europe, B. East Asia, C. Africa and D. South Asia. The maps depict the sampling locations for each population, in addition to the sample size used (n = 28 per population). Note that approximate locations for some populations (i.e. CEU, ITU and STU) are given as per the original 1000 Genomes publication[14]. Coloured tip-points below each leaf of the hierarchical clustering tree depict the geographic population label of the individual (from the maps). Hierarchical clustering was not performed on African samples due to the use of Yoruba as the outgroup population (see Methods). The bar charts show the output of the clustering algorithm ADMIXTURE applied to each sample, assuming the same number of clusters as the geographically defined samples (K = 5 for Europe, East Asia and South Asia, and K = 7 for Africa). Clustering makes no reference to the sampling locations of the individuals and is based on genetic data alone. Individuals are listed along the x-axis, grouped according to their sampling location, with bars above reflecting their cluster assignment in the following manner: each inferred cluster is labelled by a colour and the proportion of bar assigned that colour represents the probability that the individual is assigned to that cluster. Colours were manually selected (post-hoc) to match the labels in the maps. See Fig. 2 of the main text for the results of these same algorithms when applied to the NCIG + PNG dataset.

Extended Data Fig. 7 Pairwise FST between Australian, PNG and Asian populations from the SGDP.

Genetic differences between Indigenous Australian communities are significantly greater than between groups from other continents distributed over a comparable geographic range. Heatmaps show pairwise FST differences between all East Asian populations, and all Australian communities. Note, for instance, FST between Galiwin’ku and Titjikala (0.045), is as high as between Cambodia and Oroqen (0.045), groups separated by three to four times the geographical distance. All FST values were calculated on a set of variants polymorphic in an African outgroup population (Mbuti), thus providing an unbiased estimator of FST. Colour scales for both heatmaps are the same.

Extended Data Fig. 8 Comparing genetic drift shared with PNG using F-statistics.

A. p-values from a Mann Whitney U test performed pairwise between the Australian samples grouped by community. Here we assume that the outgroup F_3 statistic for each individual (relative to PNG) in Fig. 3a is drawn from a common distribution for each community. The distributions of the statistic for a pair of communities are compared using the Mann Whitney test. Significant p-values (less than 0.05; shown in red) indicate the null hypothesis, that the distributions of the statistic for each group are equal, has been rejected. A two-sided test was used, with the Bonferroni p-value adjustment method. B. Matrix of all pairwise \({{\rm{F}}}_{4}^{({\rm{T}})}\) statistics (calculated using ADMIXTOOLS) of the form \({{\rm{F}}}_{4}^{({\rm{T}})}\) (YRI, PNG; X, Y), where ‘X’ and ‘Y’ are any one of the Australian populations in the NCIG dataset. Here we separate the Tiwi samples into the islands they are sampled from. Numbers reported are Z scores (the default ADMIXTOOLS output) and are significant when they exceed +3 or −3. See Methods for description of ‘Tiwi_Outlier’ label. C. Table showing all possible F_3 statistics of the form F_3(AUAx; PNG, AUAy), where ‘AUAx’ and ‘AUAy’ (simply labelled ‘X’ and ‘Y’ in this figure) are a pair of groups from the NCIG dataset. Here we separate the Tiwi samples into the two islands, Bathurst and Melville, and we also treat the Tiwi Outlier individuals as a separate group (See Methods for a description of the Tiwi Outlier individuals and a justification for removing them from our main analyses due to evidence of substantial recent admixture with PNG in their genomes). Text within each cell is the Z-score for the F_3 statistic from a block jackknife (directly from the software Admixtools). Following the theory of Patterson et al. (2012), statistically significant evidence of admixture between PNG and AUAx, but not AUAy, is indicated by a Z-score lower than −3, here indicated by red.

Extended Data Fig. 9 Shared genetic drift between Indigenous Australian communities and a panel of Papuan populations.

(Left) Map showing the locations of all populations sampled in the dataset of Bergstrom et al. (2017), with colour code indicating the regional province. (Top Right) Scatterplot of values of outgroup F_3 statistics of the form F_3(Yoruba; Titjikala, PNG-X) versus F_3(Yoruba; Tiwi, PNG-X), where ‘PNG-X’ is a PNG individual in the dataset described by Bergstrom et al. (2017). Colours represent the sampling location of the PNG individual (see map to the left). (Bottom Right) ADMIXTURE barplot showing putative PNG (yellow) and non-PNG (purple) global ancestry estimates for each of the individuals in the above scatterplot. Individuals in the barplot are shown in the same order left to right as in the scatterplot.

Extended Data Fig. 10

Mitochondrial phylogenetics. Population Mitochondrial DNA phylogeny of all individuals from the NCIG + PNG dataset, plus additional sequences from GenBank (see Methods and Supplementary Note 7 for samples used and phylogenetic methods). Tip-point labels indicate the community the individual was sampled from. Coloured circles over nodes indicate coalescence events between PNG and Indigenous Australian haplotypes which date to within the last ~35 ka. Clade labels of sub-lineages (P3, N13 and Q2) mark the lineages involved.

Extended Data Fig. 11 Map with pie charts showing frequencies of the P3, N13 and Q2 Mitochondrial haplogroups.

Map with pie charts showing frequencies of the three haplogroups (P3, N13 and Q2), with recent (~35 ka) TMRCA to Melanesian sister lineages in Indigenous Australian communities from both the NCIG dataset, and previously published studies. Note the apparent enrichment of these haplogroups in the Top End and Kimberley regions of Australia. The P3 haplogroup frequency was scored instead of P3b, as some studies did not genotype to this degree of resolution. The P3 lineage coalesces approximately 35 ka and contains both PNG and Indigenous Australian sub-lineages.

Extended Data Table 1 Per population sample count of autosomal SNVs at VQSR = 99.8 before (top) and after (bottom) ancestry masking

Supplementary information

Supplementary Information

Supplementary Notes 1–7, including Figs. 1–6, Tables 1–3 and additional references.

Reporting Summary

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Silcocks, M., Farlow, A., Hermes, A. et al. Indigenous Australian genomes show deep structure and rich novel variation. Nature 624, 593–601 (2023). https://doi.org/10.1038/s41586-023-06831-w

- Received: 29 November 2022

- Accepted: 03 November 2023

- Published: 13 December 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06831-w

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

- Indigenous diversity in Australia’s DNA tapestry unlocked for genetic equity

- Katrina G. Claw

- Amber Nashoba

Nature (2023)

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

The landscape of genomic structural variation in Indigenous Australians

Article Open access 13 December 2023

Andre L. M. Reis, Melissa Rapadas, … Ira W. Deveson

High-depth African genomes inform human migration and health

Article 28 October 2020

Ananyo Choudhury, Shaun Aron, … Neil A. Hanchard

Whole genomes from Angola and Mozambique inform about the origins and dispersals of major African migrations

Article Open access 02 December 2023

Sam Tallman, Maria das Dores Sungo, … Sandra Beleza

The landscape of genomic structural variation in Indigenous Australians

- Andre L. M. Reis

- Melissa Rapadas

- Ira W. Deveson

Nature Article Open Access 13 Dec 2023

Indigenous diversity in Australia’s DNA tapestry unlocked for genetic equity

- Katrina G. Claw

- Amber Nashoba

Nature News & Views 13 Dec 2023

Article Open access Published: 13 December 2023

The landscape of genomic structural variation in Indigenous Australians

Andre L. M. Reis,

Melissa Rapadas,

Jillian M. Hammond,

Hasindu Gamaarachchi,

Igor Stevanovski,

Meutia Ayuputeri Kumaheri,

Sanjog R. Chintalaphani,

Duminda S. B. Dissanayake,

Owen M. Siggs,

Alex W. Hewitt,

Bastien Llamas,

Alex Brown,

Gareth Baynam,

Graham J. Mann,

Brendan J. McMorran,

Simon Easteal,

Azure Hermes,

Misty R. Jenkins,

The National Centre for Indigenous Genomics,

Hardip R. Patel &

…

Ira W. Deveson

Show authors

Nature volume 624, pages 602–610 (2023)

- 9297 Accesses

- 1 Citations

- 93 Altmetric

- Metrics details

Abstract

Indigenous Australians harbour rich and unique genomic diversity. However, Aboriginal and Torres Strait Islander ancestries are historically under-represented in genomics research and almost completely missing from reference datasets1,2,3. Addressing this representation gap is critical, both to advance our understanding of global human genomic diversity and as a prerequisite for ensuring equitable outcomes in genomic medicine. Here we apply population-scale whole-genome long-read sequencing[4] to profile genomic structural variation across four remote Indigenous communities. We uncover an abundance of large insertion–deletion variants (20–49 bp; n = 136,797), structural variants (50  b–50 kb; n = 159,912) and regions of variable copy number (>50 kb; n = 156). The majority of variants are composed of tandem repeat or interspersed mobile element sequences (up to 90%) and have not been previously annotated (up to 62%). A large fraction of structural variants appear to be exclusive to Indigenous Australians (12% lower-bound estimate) and most of these are found in only a single community, underscoring the need for broad and deep sampling to achieve a comprehensive catalogue of genomic structural variation across the Australian continent. Finally, we explore short tandem repeats throughout the genome to characterize allelic diversity at 50 known disease loci[5], uncover hundreds of novel repeat expansion sites within protein-coding genes, and identify unique patterns of diversity and constraint among short tandem repeat sequences. Our study sheds new light on the dimensions and dynamics of genomic structural variation within and beyond Australia.

Similar content being viewed by others

Main

Australia is home to hundreds of Aboriginal nations or clans who inhabited all geographical regions throughout the continent, prospering in their diverse environments for at least 50,000 years6,7,8,9,10. More than 250 distinct languages were spoken at the time of invasion by people from Europe and around 150 of these survive today[11]. Australian Indigenous communities practice cultures that are among the world’s oldest continuous surviving cultures. These are highly varied, but commonly emphasize the importance of kinship, ancestry and relationships to the landscape and environment[10].

Whereas the remarkable cultural and linguistic diversity of Indigenous Australians is well documented, their rich and unique genomic diversity is relatively unexplored. Indigenous peoples have been historically under-represented in genomics research globally and Aboriginal ancestries are currently absent from leading international genomics resources, including the 1000 Genomes Project and gnomAD reference databases1,2, as well as the recent draft Human Pangenome Reference[3]. Such resources are central to the interpretation, diagnosis and treatment of genetic disease, but have reduced utility for communities without appropriate representation[12]. There is a pressing need to close this Indigenous representation gap to ensure equitable outcomes from genomic medicine in Australia13,14. Moreover, as one of the six inhabited continents on earth, the current lack of genomic data from Australia is a major gap in our understanding of global human genomic variation.

The National Centre for Indigenous Genomics (NCIG) aims to address this gap, by engaging Aboriginal and Torres Strait Islander communities in genomics research (https://ncig.anu.edu.au/). The NCIG has developed frameworks for Indigenous genomics that prioritize community leadership, participation and data sovereignty15,16. The NCIG aims to develop relationships of deep trust with Indigenous communities, cataloguing their genomic diversity and conducting research in a manner that is sustainable, ethical, and not only beneficial to the partner communities but also aligned with their own ways of knowing15,16.

In this study, we performed population-scale long-read sequencing using Oxford Nanopore Technologies (ONT) instruments for four NCIG-partnered Aboriginal communities across northern and central Australia, as well as non-Indigenous Australians. The use of long-read sequencing technology, in combination with the recently completed telomere-to-telomere human reference genome[17] (T2T-chm13), enables us to explore uncharted Aboriginal genomic variation. Long reads can resolve repetitive or non-unique genes and regions that are intractable with dominant short-read sequencing platforms[18]. Long reads are also superior for the detection of structural variants (SVs), which account for the majority of the differences between the genomes of any 2 individuals and at least 25% of their deleterious alleles, yet are poorly understood owing to technical and analytical limitations18,19,20.

Here we apply long-read sequencing[4] at population scale to a diverse Indigenous cohort. We begin to describe the landscape of genomic structural variation in Indigenous Australians and establish frameworks for interpreting this variation in the context of genomic medicine.

ONT sequencing in Aboriginal communities

To explore genomic structural variation in Indigenous Australians, we performed whole-genome ONT sequencing on individuals from four remote Aboriginal communities with whom the NCIG has developed partnerships: Tiwi Islands (Wurrumiyanga, Pirlangimpi and Millikapiti communities; NCIG-P1), Galiwin’ku (NCIG-P2), Titjikala (NCIG-P3) and Yarrabah (NCIG-P4). These span a wide geographic, cultural and linguistic landscape (Fig. 1a). We sequenced between 9 and 41 individuals from each community, 121 in total. We also sequenced 18 non-Indigenous Australian individuals of European ancestry for comparison, and two reference individuals from the Genome in a Bottle project[21] (HG001, HG002; Supplementary Table 1). High molecular weight DNA was extracted from saliva or blood and sequenced on an ONT PromethION device (see Methods). Non-human reads present in saliva samples were identified and showed negligible rates of erroneous alignment to human chromosomes (Extended Data Fig. 1a,b). We obtained a median of 30-fold (range 14–47) genome coverage and 9.2 kb (2.7–16.8 kb) read length N50 (Extended Data Fig. 1c,d). Although DNA samples varied in quality, we obtained a minimum of 10-fold coverage in reads of at least 5 kb for every individual, providing a strong foundation for profiling genomic structural variation across the cohort (Fig. 1b and Extended Data Fig. 1e).

Fig. 1: Long-read sequencing in Indigenous Australian communities.

a, Study design and analysis workflow. DNA samples were collected from four Indigenous communities: Tiwi Islands (NCIG-P1), Galiwin’ku (P2), Titjikala (P3) and Yarrabah (P4), and from unrelated European individuals (non-NCIG). The map shows geographic locations, with population sizes and participant numbers underneath. ONT sequencing was performed and reads aligned to the T2T-chm13 genome. SVs were called for each individual, then joint calling was performed to generate a non-redundant set of SVs, genotyped for each individual. SVs were characterized by type, size and context and compared to existing SV datasets. SVs were compared between individuals and communities, with non-NCIG individuals as an outgroup. Short tandem repeat (STR) alleles were genotyped to assess variation. Chr, chromosome; DEL, deletions; INS, insertions; ME, mobile elements. b, Average genomic coverage as sequencing reads were filtered by a minimum read-length cut-off. Each line represents one individual. Pie charts show the proportion of male and female participants from each community. c, Percentage of genome with zero coverage for Illumina short-read and ONT long-read libraries from HG001 and HG002, aligned to either hg38 or T2T-chm13. d, Percentage of genome covered by alignments with low mapping quality (MAPQ < 5). e, Number of SVs detected.

Publication of the first complete human genome[17] was a landmark for the field, but so far there are few major studies outside the T2T Consortium that have used T2T-chm13 as their chosen reference genome. We evaluated mappability and structural variant (SV) detection against the T2T-chm13 reference, by comparison to hg38, using both ONT and short-read sequencing data from the HG001 and HG002 reference samples (Methods). As expected, ONT data exhibited superior unique-alignment coverage and more comprehensive SV detection than short reads (Fig. 1c–e). These advantages were further enhanced by use of the T2T-chm13 reference, which had proportionally fewer regions of zero coverage (mean 4.9% versus 10.0%) or low mappability (MAPQ < 5; mean 0.2% versus 1.4%) and, as a result, afforded an additional approximately 125 Mb of total reference sequence that was accessible to analysis with ONT data (Fig. 1c,d). ONT sequencing depth had negligible effect on SV detection above a threshold of approximately 20X coverage, consistent with independent benchmarking[22] (Extended Data Fig. 2a). Manual inspection of medically relevant repetitive genes, such as MUC1[23] (Extended Data Fig. 2b), showed these were generally best resolved using the combination of ONT and T2T-chm13. Together, these results highlight the advantages of long-read sequencing and T2T-chm13 for profiling genomic structural variation at high resolution.

Adopting T2T-chm13 as our genome reference, we called large insertion–deletions (indels) (20–49 bp) and SVs (50 bp–50 kb) in each individual (CuteSV[24]; Fig. 1a). Variants were filtered to exclude events with weak evidence (QUAL ≥ 5). We detected 21,723 SVs, on average, per individual, of which 19,089 were retained after filtering (Extended Data Fig. 1f). The retained SV count is somewhat lower than reported in several recent long-read sequencing studies25,26, reflecting our preference for retaining only high-confidence SVs. Callsets were then merged (using Jasmine[27]) into a unified joint-call catalogue comprising 159,912 unique SVs and 136,797 large indels (Fig. 1a and Methods). Notably, this surpasses the 134,886 SVs recently identified by ONT sequencing of 3,622 Icelanders[25] (the largest long-read sequencing study published to date), reflecting higher genetic heterogeneity in our smaller cohort. We additionally applied a read-depth method (CNVPytor[28]) to identify large copy number variants (CNVs) (>50 kb) in each individual. We identified 11 high-confidence (P < 10−4) CNVs per individual (9 deletions, 2 duplications) on average, which were merged into 156 unique regions of variable copy number across the cohort (Extended Data Fig. 3a).

Genomic structural variation landscape

To better characterize the landscape of genomic structural variation, we next stratified variants by type, size and context (Fig. 1a and Methods). A clear majority of all non-redundant variants (84.9%) were composed of repetitive sequences, including 103,425 STR (2–12 bp) and 123,667 tandem repeat (TR) (>12 bp) expansions and contractions, and 25,096 insertions or deletions of interspersed mobile element sequences (Fig. 2a). The remaining 37,574 variants (12.6%) were found to be non-repetitive (Fig. 2a). Although CNVs were few in number, they encompassed >65 Mb of genome sequence across the entire cohort, with an average of 2.8 Mb per individual (Extended Data Fig. 3b). We observed deletions up to 13 Mb (mean 243 kb) and duplications up to 1.8 Mb (mean 303 kb; Extended Data Fig. 3c,d).

Fig. 2: Landscape of genomic structural variation.

a, Number of non-redundant variants identified across the cohort (n = 141). Large indels (left; 20–49 bp) and SVs (right; ≥50 bp) are shown separately and parsed by type: non-repetitive, tandem repeats (homopolymer (HOMO), STR and TR) and mobile elements (long terminal repeat (LTR), long interspersed nuclear elements (LINE), short interspersed nuclear element (SINE), retroposon and DNA/DNA transposon (DNA)). Light shades represent deletions and dark shades represent insertions. b, Frequency of non-redundant SVs relative to distance from the nearest telomere. c, Size distribution of insertions (positive values) and deletions (negative values), parsed by type (colour scheme as in a). Characteristic peaks for Alu elements (280 bp) and L1 elements (6 kb) are marked. d, Size distributions for each variant type. Mobile element SVs are classified as ‘complete’ if they encompass one or more complete annotated element or ‘fragment’ if they encompass only part of an annotated element. e, Number of non-redundant SVs found in a search performed against a combination of the gnomAD, deCODE and HGSVC (freeze 4) SV annotations. SVs were first lifted from T2T-chm13 to hg38. Some could not be lifted because they were deleted or partially deleted in hg38. SVs that could be lifted were categorized as ‘annotated’ or ‘unannotated’ on the basis of reciprocal overlap to any single annotated SV. High (>80%) and moderate (50–80%) overlaps were considered as annotated, whereas low (<50%; beige) and no overlap were considered as unannotated.

Structural variation was not distributed evenly across the genome, but showed higher density within approximately 5 Mb of the telomere on each chromosome (Fig. 2b), as has been reported elsewhere25,29. This effect was almost entirely driven by TR-associated SVs, which were enriched in sub-telomeric regions, with other classes being evenly distributed (Extended Data Fig. 4a). Both metacentric and acrocentric chromosomes were similarly affected (Extended Data Fig. 4b).

We observed characteristic differences in size between variants of different types (Fig. 2c,d). TR-associated SVs were generally larger than STRs or non-repetitive SVs. Size distributions for mobile element SVs displayed clear peaks around expected sizes for major repeat families, including Alu (a SINE of approximately 280 bp), L1 (a LINE of approximately 6 kb) and SVA[30] (SINE-R/VNTR/Alu; a retroposon of approximately 2 kb) (Fig. 2c,d). Whereas most SVs associated with mobile elements encompassed only part of an annotated element, the aforementioned peaks are formed by SVs encompassing one or more complete elements, which represent transposition events occurring since the common ancestor of individuals in our cohort and the T2T-chm13 reference (European origin; Fig. 2d). Among these complete elements, SINEs were dominant (n = 8,867), reflecting comparatively high Alu activity, and significant numbers of LINE (n = 501) and retroposon (n = 327) transpositions were also detected (Fig. 2a,d).

Given the inclusion of unique, under-represented Australian communities and the use of long-read sequencing, our catalogue contained a high proportion of SVs that have not been previously annotated (Fig. 2e). We compared our SV callset to: (1) the gnomAD SV database[19], which spans a diverse global cohort sequenced on short-read platforms; (2) an SV annotation published recently by deCODE genetics[25], based on population-scale ONT sequencing of Icelandic individuals; and (3) a state-of-the-art long-read SV annotation based on 35 diverse individuals analysed by the Human Genome Structural Variation Consortium (HGSVC). For this analysis, it was necessary to first convert SV coordinates to the hg38 reference, on which these annotations are based (using LiftOver). A significant number of SVs could not be lifted from T2T-chm13 to hg38 because their corresponding positions were fully (24.9%) or partially (18.3%) missing from the latter (Fig. 2e). Of the 90,578 out of 159,912 SVs that were successfully lifted to hg38, we found a highly similar annotated SV (more than 80% reciprocal overlap) for 37,421 SVs and an annotated SV at the same position with moderate similarity (50–80%) for 22,625 SVs (Fig. 2e). The latter were especially common for TR- and STR-associated SVs, where alternative alleles at variable TR or STR loci often appear as partially overlapping SVs (Extended Data Fig. 4c,d). Together, this shows that there is an annotated SV for around 38% of all non-redundant SVs in our callset (Fig. 2e), with the remaining SVs that were successfully lifted having low (<50%; n = 8,770) or no (0%; n = 21,762) overlap with any annotated variant. Because SVs that could not be lifted to hg38 cannot be assessed in this manner, we instead provide an upper-bound novelty estimate of 62% (assuming non-lifted SVs are all novel) and a lower-bound estimate of 19%.

Distribution and diversity

We next assessed the distribution of genomic structural variation among Indigenous and non-Indigenous individuals in the cohort. Overall, the majority of all non-redundant SVs were either private (that is, found in a single individual; 26.3%) or polymorphic (found in less than 50% of individuals; 65.5%), with the remaining being classified as major (found in more than 50% of individuals; 7.8%) or shared alleles (found in all individuals; 0.2%) (Extended Data Fig. 5b). Although different SV types were distributed uniformly among individuals and communities (Fig. 3a and Extended Data Fig. 5a), they varied in the degree to which they were shared between individuals (Fig. 3b and Extended Data Fig. 5c). For example, the proportional representation of STR- and TR-associated SVs was skewed towards polymorphic and private variation, whereas mobile elements and non-repetitive SVs were proportionally enriched among major and shared variation (Fig. 3b). These trends indicate the different rates at which different classes of SVs emerge and change over generations.

Fig. 3: Distribution of SVs in Indigenous and non-Indigenous individuals.

a, Number of SVs identified in individuals from each group, parsed by type. Colour scheme as in b. b, Proportional representation of different types for SVs identified within a given number of individuals (degree of sharedness). SVs were labelled as private (1 individual), polymorphic (more than one individual and less than 50% of individuals), major (≥50%, but not all individuals) and shared (all individuals). c, Proportion of SVs in each individual that were found exclusively in NCIG individuals (NCIG-only), or exclusively in non-NCIG individuals (NCIG-absent), or across both (global). Colour scheme as in d. d, Proportional representation of NCIG-only, NCIG-absent and global SVs according to degree of sharedness.

A large proportion of SVs across the complete non-redundant catalogue was seen only among Indigenous individuals (NCIG-only; 48.5%) or only among non-Indigenous participants (NCIG-absent; 9.2%) (Fig. 3c). NCIG-only SVs made up a significantly higher proportion of total SVs in a given Indigenous individual (15.0 ± 2.0%) than NCIG-absent SVs in non-Indigenous individuals (5.2 ± 0.4%) (Fig. 3c and Extended Data Fig. 6a). The majority of NCIG-only variants were polymorphic (Fig. 3d and Extended Data Fig. 6b) and were previously unannotated—more so than for NCIG-absent variants (Extended Data Fig. 6c). On average, each Indigenous individual harboured 2,884 ± 520 NCIG-only SVs, of which 311 ± 259 were unannotated (lower-bound estimate) and may therefore represent exclusively Australian Indigenous variation.

The clear genetic distinctions between Indigenous Australian and non-Indigenous Australian individuals was further reiterated by principal coordinate analysis (PCOA) and fixation index (FST) analysis of structural variation (Fig. 4a and Extended Data Fig. 7a). This also highlighted the distinct genetic architecture of different communities, which formed largely separate PCOA clusters (Fig. 4a). Indeed, among Indigenous individuals, we found that 56.4% of NCIG-only SVs were found in just a single individual or community, whereas NCIG-only SVs shared between more than one individual across all communities were relatively rare (2.8%) (Fig. 4b and Extended Data Fig. 7b). This was corroborated by independent analysis of SNVs detected with short-read sequencing data from the same NCIG partner communities, with similar proportions of continent- and community-specific variation being observed (Extended Data Fig. 7c,d). Shared SVs showed a proportional enrichment of mobile elements and a depletion of TR SVs (Extended Data Fig. 8a), consistent with the contrasting polymorphism for these SV types (Fig. 3b). Of the approximately 311 exclusively Indigenous SVs in a given individual identified above, an average of around 185 ± 31 were not found outside their community.

Fig. 4: Distribution of SVs between Indigenous communities.

a, PCOA representing the distance between individuals in the cohort based on their SV compositions. The percentage of variance is indicated in parentheses for each principal coordinate axis (PCOA1 and PCOA2). Individuals are coloured according to their group. b, Distribution of NCIG-only variants (Fig. 3) shared among the four NCIG communities. SVs were classified as private (n = 1 individual), community-specific (n > 1 individual in 1 community), widespread (n > 1 individual in more than 1 community) or shared (n > 1 individual in all 4 communities). c, Proportion of private, community-specific, widespread and shared NCIG-only variants among individuals, grouped by community. A total of n = 141 individuals (41 NCIG-P1, 32 NCIG-P2, 9 NCIG-P3, 39 NCIG-P4 and 20 non-NCIG individuals) were examined. The centre line shows the median, box edges delineate the interquartile range (IQR) and whiskers extend to 1.5× IQR from the hinge. d, SV discovery curve in which, starting with a single NCIG individual, the number of new non-redundant SVs is counted as new individuals are iteratively added. SVs shared among all previously added samples are shown as green portions of each bar.

Next, we generated discovery curves that model the diversity of structural variation within a set of individuals (Methods). Across the 121 Indigenous individuals in the cohort, cumulative SV discovery did not approach saturation, indicating many further SV alleles remain to be sampled (Fig. 4d and Extended Data Fig. 8b–d). This diversity was not shared equally among different communities or variant types (Fig. 4c and Extended Data Fig. 8e). Individually, the Tiwi Islands (NCIG-P1), Galiwin’ku (NCIG-P2) and Titjikala (NCIG-P3) communities each showed lower within-community SV diversity than seen among the non-NCIG comparison group, reflecting their small population sizes and relative isolation (Fig. 1a). By contrast, Yarrabah (NCIG-P4) harboured substantially higher genomic diversity than the other communities, a higher proportion of private variation, and alone showed greater diversity than the non-Indigenous group (Fig. 4c and Extended Data Fig. 8b). NCIG-only SVs showed greater heterogeneity among NCIG individuals than for NCIG-absent SVs among non-NCIG individuals (Extended Data Fig. 8b). Finally, we found that SV diversity was driven most strongly by TR-associated SVs, whereas new discovery of mobile elements and non-repetitive SVs was largely saturated (Extended Data Fig. 8b,d).

Functional context

Given the vast diversity of genomic structural variation described above and the predominance of SV classes that are poorly studied, we next used measures of purifying selection to investigate their functional relevance. A large indel or SV intersecting with one or more coding sequence (CDS) exons in a protein-coding gene is likely to truncate or alter its open reading frame, whereas a variant within an intron, untranslated region (UTR) or proximal gene-regulatory region may affect transcription, translation or splicing. The extent to which these events disrupt gene function should be modelled by depletion of structural variation within essential genes among otherwise healthy populations[19].

Across our complete cohort (n = 141), we detected 126,473 non-redundant variants (58,079 indels and 68,394 SVs) intersecting protein-coding loci, including 1,462 affecting CDS exons. An average individual possessed 156 ± 15 CDS variants and 20,124 ± 1,308 within non-CDS regions of protein-coding genes (introns, UTRs and proximal regulatory regions) (Extended Data Fig. 9a). There was an enrichment of private variants intersecting CDS regions (33.7%) compared with the proportion of private variants (24.1%) in intergenic regions, consistent with purifying selection. Variants intersecting CDS exons were almost all either non-repetitive (33.7%) or TR-associated (59.5%), with a strong depletion of STR and mobile element SVs in CDS, relative to intronic and intergenic regions (Fig. 5a). We also identified 82 large CNVs that constituted whole-gene deletions (n = 225 genes) or whole-gene duplications (n = 237 genes) across the cohort (Extended Data Fig. 9b,c).

Fig. 5: Functional relevance of genomic structural variation.

a, Proportion of variant types identified within CDS exons, non-CDS regions of protein-coding genes (introns, UTRs and ±2 kb proximal regulatory regions) and intergenic regions, for large indels (left; 20–49 bp) and SVs (right; ≥50 bp). Variants are classified as: non-repetitive, STR, TR and mobile element (fragment or complete). b, Left, variant density in CDS and non-CDS regions of protein-coding genes in different LOEUF deciles, which quantify intolerance to loss-of-function variation (first decile corresponds to the highest constraint). b, Right, variant density in non-CDS parsed by variant type and normalized to their 10th decile. c, Sequence bar chart shows ATXN3 STR alleles, including 20 bp of upstream flanking sequence, genotyped for every individual (two alleles each). Dashed lines show STR size cut-offs for intermediate and fully pathogenic alleles. A single pathogenic allele was identified in an NCIG-P2 individual. All others are in the ‘normal’ range.

We next parsed protein-coding genes according to loss of function observed/expected upper-bound fraction (LOEUF), a metric that quantifies their intolerance to loss-of-function variation, developed previously by gnomAD[2]. This approach revealed clear constraint on structural variation in CDS regions (Fig. 5b). For example, we saw an approximately 12-fold reduction in the size-normalized density of structural variation among the most essential genes (LOEUF decile 1; 1.29 × 10−5) compared with the least essential genes (LOEUF decile 10; 1.60 × 10−4) (Fig. 5b and Methods). We also observed significant, albeit weaker, constraint on non-CDS variants, with an approximately 1.5-fold difference in variant frequency between the highest and lowest deciles (Fig. 5b). Mobile element-associated SVs showed the strongest purifying selection (1.8-fold difference between highest and lowest deciles) and STR-associated SVs showed the weakest (1.3-fold) (Fig. 5b). Assessing LOEUF among Indigenous-specific variation, we found that most SVs (81.1%) in CDS regions of essential genes were private or community-specific (Extended Data Fig. 9d).

Since evidence of selection is critical for interpreting the functional relevance of genetic variation, the findings above help to establish the suitability of our analysis framework and SV catalogue to inform genomic medicine applications in Indigenous Australians. Even among just 141 individuals sequenced here, we identified 69 deletions or insertions affecting CDS regions of genes in LOEUF deciles 1 and 2, 44 of which were not previously annotated. This includes complete or near-complete deletions of essential genes including PRKRA, BRD9, SHOX, NID2, PABPC5, JAZF1, NIPA2 and ANOS1, and a large somatic CNV deletion (13 Mb) in a non-NCIG individual affecting three genes in LOEUF decile 1 (ARHGAP42, YAP1 and DCUN1D5).

One notable SV in an essential gene (LOEUF decile 1) was a 130 bp CAG STR expansion in ATXN3 that is known to cause Machado–Joseph Disease[31] (MJD, also known as spinocerebellar ataxia type 3). This pathogenic allele was detected in a single individual from Galiwin’ku community (NCIG-P2), with long-read sequencing clearly defining the size, position and sequence of the STR expansion, unlike short-read whole-genome sequencing on the same individual (Fig. 5c and Extended Data Fig. 9e). MJD is a late onset, progressive movement disorder with autosomal dominant inheritance and complete penetrance for expansions of this size[31]. MJD affects around 5 out of every 100,000 people worldwide, but is estimated to be more than 100 times more prevalent among Indigenous populations in some areas of Australia’s Northern Territory[32].

The individual in question had consented to receive reportable genetic findings arising from NCIG research. This prompted an ongoing dialogue between NCIG, Galiwin’ku representatives, local genetic counsellors and the MJD foundation (https://mjd.org.au/), who work with remote Northern Territory Aboriginal communities to develop unique clinical genetics service models tailored for their needs[33]. Under recommendation of the MJD Foundation, genetic counsellors were able to contact the individual and their family, arranging for clinical testing and appropriate follow-up.

STR expansions

The preceding example highlights the utility of long-read sequencing for profiling variation in STR sequences—both normal and pathogenic—as well as the importance of ancestry in interpreting this variation. STRs are highly polymorphic, and STR expansions are causative pathogenic variants in at least 37 neurogenetic and 10 congenital disorders[5]. However, STR expansions are refractory to analysis with short-read sequencing and, as a result, have been relatively poorly characterized to date, particularly among minority communities, such as Indigenous Australians. Our dataset provides a unique opportunity to explore allelic diversity in STR sequences at high-resolution and population scale.

Across the cohort, we detected 55,595 non-redundant variants that constitute STR expansions (that is, insertions) and 47,830 STR contractions (that is, deletions;  Methods) relative to the T2T-chm13 reference. These ranged in size from approximately 20 bp (our lower cut-off) to 99,204 bp in size and occurred predominantly in intergenic (63.9%) or non-CDS gene regions (35.9%) (Fig. 6a). STR period size was negatively correlated with the global frequency of STR expansions and contractions. However, trinucleotide and hexanucleotide repeats were outliers from this trend, showing markedly lower frequencies than other periods (Fig. 6a). The opposite was true within CDS regions, where in-frame expansions (that is, 3 bp, 6 bp, and so on) occurred at higher frequencies than other periods (Fig. 6a). Therefore, although in-frame expansions and contractions are more tolerated within coding sequences (because they do not cause frameshifts), there appears to be higher constraint on in-frame STRs across the remainder of the genome. We speculate that this acts to limit the potential for spurious expression of toxic homomeric polypeptides that contribute to pathogenicity in many STR disorders[5].

Fig. 6: Landscape of STR expansions.

a, Top, length distribution of all non-redundant TR and STR insertions (that is, expansions) detected across the cohort (n = 141), broken down by period size. Bottom left, number of non-redundant STR expansions in each period. Bottom centre, relative frequency of each period within CDS exons. Bottom right, proportional composition of each period based on genomic context. b, Frequency of every possible triplet (top) and pentanucleotide (bottom) motif among non-redundant STR expansions. Selected motifs known to cause different repeat disorders are highlighted. For triplet disorders, pathogenic motifs with gain-of-function mechanisms are shown in red, those with loss-of-function mechanisms are shown in blue. c, Left, normalized standard deviation (range 0–1) of allele sizes observed within each community group (matrix columns) for different STR sites (matrix rows). All expanded sites of period ≥3 bp within protein-coding genes, in which allelic composition was significantly different between groups are shown (one-way ANOVA, P < 0.05). Hierarchical clustering groups STR sites on the basis of patterns of variability between groups. c, Right, all STR alleles (two per individual) for three example sites showing distinct patterns of variation. BLOC1S2 (top) has an intronic STR with higher allelic diversity in NCIG versus non-NCIG individuals. AC012531 (middle) has lower allelic diversity in NCIG versus non-NCIG individuals. PRRC2B (bottom) shows community-specific patterns, with NCIG-P1 and NCIG-P4 exhibiting heterogeneity.

Consistent with this idea, expansions of the STR motifs associated with poly-glutamine (CAG) and poly-glycine (CGG) disorders, which include MJD and a range of other disorders, were globally rare by comparison to other motifs[5] (Fig. 6b). In contrast to these dominant gain-of-function disorders, the GAA expansion motif that triggers epigenetic silencing of FXN (that is, loss of function) in Fredrich’s ataxia, was relatively common, suggesting expansions of this motif are not typically deleterious in other contexts[5] (Fig. 6b).

Besides triplets, intronic pentanucleotide STRs are most widely associated with known disorders[5]. In this context, it was notable that pentanucleotide STR expansions occurred at lower frequency (approximately 17%) within intronic regions than for other period sizes (around 40%), indicative of context-specific constraint (Fig. 6a). However, in contrast to triplet disorders, motifs known to be associated with disease, such as AAGGG (CANVAS (cerebellar ataxia with neuropathy and vestibular areflexia syndrome)), TTTCA (FAME (familial adult myoclonic epilepsy)) and TGGAA (SCA31 (spinocerebellar ataxia type 31)), were among the most frequent pentanucleotide expansions (Fig. 6b).

To better resolve the STR landscape, we performed diploid STR genotyping, expansion discovery and visualization[34]. We focused our analysis on 685 STR sites of period 3 bp or larger within protein-coding loci (including 7 within CDS exons) that were significantly expanded in at least one individual (Extended Data Fig. 10a), as well as 50 known disease-associated STR loci[5]. Using this approach, we stratified STR sites according to allelic diversity across communities, identifying 231 sites (18.9%) that showed inter-community differences in allelic composition (Fig. 6c and Extended Data Fig. 10b). We found 155 sites that were more diverse in Indigenous than non-Indigenous individuals and 76 sites where the opposite was true (Fig. 6c). Many STRs showed more local effects, such as increased allelic diversity in just a single community, or expansions that were limited to a single individual or a small number of individuals (Fig. 6c).

Discussion

Our study is a major genomic survey of Indigenous Australians. Previous publications that include genomic data from Aboriginal and/or Torres Strait Islander peoples have largely focused on historical demographic processes8,35,36,37,38. These relied on mitochondrial38,39 or short-read whole-genome sequencing8,35,36,37. By contrast, we used whole-genome long-read sequencing to generate data that are suitable for exploring the landscape of Indigenous genomic structural variation.

We found a diversity of structural variation across four remote Aboriginal communities in northern and central Australia. This was predominantly repetitive, encompassing thousands of tandem repeats and mobile elements. A significant proportion was found only in Indigenous individuals in our study, and have not been previously annotated in diverse global reference data19,25. However, exclusively Indigenous variants were generally not shared throughout the continent.

Our study sheds new light on the rich and unique genetic diversity of Indigenous Australians. Owing to the long history of continuous occupation, Australia’s Indigenous peoples are highly genetically distinct from non-Australians. This underscores the need for ancestry-appropriate reference data for genomic medicine, of which there is a shortage12,13,14. Moreover, Indigenous Australians should not be viewed as genetically homogenous. We show that different communities, clans and/or nations have highly distinct genomic architectures, mirroring their cultural and linguistic diversity[10]. Therefore, broad engagement—far beyond the four communities we have profiled here—will be required to adequately survey Indigenous genomic variation and, ultimately, to achieve equitable outcomes in genomic medicine.

Our study is among a small number of recent efforts to implement long-read sequencing at population scale[4], and others have so far focused on comparatively homogeneous, well-studied populations (for example, Icelandic[25], Chinese[40] and Japanese[41] populations). Although our cohort was smaller than each of these studies, we identified a greater number of total non-redundant SVs, reflecting higher genetic heterogeneity among our participants. We use this rich catalogue to articulate a number of fundamental insights into the landscape of genomic structural variation in human populations that reach beyond Australia. For example, we show that: (1) SVs are predominantly repetitive, with TRs, STRs and mobile elements underpinning around 87% of SVs per individual; (2) SVs of different types and sizes show clear differences in their dynamics of inheritance, with TR and STR SVs being more polymorphic than mobile elements and non-repetitive SVs; (3) SVs in both CDS and non-CDS regions of protein-coding genes are under purifying selection and different SV types show different signatures of constraint.

Our high-resolution survey of allelic diversity among STRs is similarly informative. We uncover an abundance of STR variation across the genome. STRs show distinct, context-specific signatures of selection, with specific periods and motifs showing elevated constraint, globally. For both novel expansion sites and known disease-associated loci, we show pervasive Indigenous versus non-Indigenous and inter-community differences in STR allele composition. Constructing a clear picture of this complex background of normal STR variation is critical for the discovery and diagnosis of STR expansion disorders[34]. However, most existing reference data are based on European and East Asian cohorts and are therefore of reduced suitability for Indigenous Australian communities. Tangible examples of local effects are provided by a unique STR motif that causes CANVAS in individuals of Māori descent[42] and MJD, an expansion disorder with markedly high frequency in Northern Territory Aboriginal communities[32]. Our study begins to establish the appropriate context for interpreting Indigenous STR variation in future genomic medicine initiatives. Indeed, the unexpected identification of a pathogenic MJD expansion in one individual highlights the strength of our approach in this domain.

Ethics and inclusion

We are indebted to the individuals and their communities who participated in this research and to the NCIG Indigenous-majority Governance Board who helped guide this work in a culturally appropriate manner. The research was conducted in accordance with core principles of Indigenous community engagement, leadership and data sovereignty, as set out in the NCIG governance framework, approved under the Australian federal legislation (https://ncig.anu.edu.au/files/NCIG-Governance-Framework.pdf).

Saliva and/or blood samples were collected from consenting individuals among four NCIG-partnered communities: Tiwi Islands (comprising the Wurrumiyanga, Pirlangimpi and Millikapiti communities), Galiwin’ku, Titjikala and Yarrabah, between 2015 and 2019. This study was approved by the Australian National University Human Research Ethics Committee (Ethics protocol number 2015/065). Non-Indigenous comparison data, generated from unrelated Australian individuals of European ancestry, was drawn from two existing biomedical research cohorts: the Tasmanian Ophthalmic BioBank (Ethics protocol number 2020/ETH02479) and the Australian and New Zealand Registry of Advanced Glaucoma (Southern Adelaide Clinical Human Research Ethics Committee approval 305-08).

Methods

Cohorts

Saliva and/or blood samples were collected from consenting individuals among four NCIG-partnered communities: Tiwi Islands (comprising the Wurrumiyanga, Pirlangimpi and Millikapiti communities), Galiwin’ku, Titjikala and Yarrabah, between 2015 and 2019. Non-Indigenous comparison data, generated from unrelated Australian individuals of European ancestry, was drawn from two existing biomedical research cohorts: the Tasmanian Ophthalmic BioBank, and the Australian and New Zealand Registry of Advanced Glaucoma.

Saliva sample collection and DNA extraction

Saliva samples were collected in Oragene DNA collection tubes (OG-500, DNA Genotek). Individuals were requested to avoid food intake 30 min prior to the collection and were asked to fill the collection tube to the best of their capacity. Approximately 3 ml of total material (including stabilizing liquid) was collected from individuals. Samples were transported to the NCIG lab in checked-in baggage in flight at the end of community visits. Saliva tubes were subject to large changes in temperature during collection and transport, and were kept at room temperature or at 4 °C until further processing. Samples were split into 2 or more aliquots of 1 ml each depending on the quantity of material available after heating samples at 50 °C for 2 h, as recommended by the manufacturer. Each sample tube was separately processed in a hood to reduce handling errors, cross contamination and external contamination. One of the aliquots with 1 ml sample was used for the DNA extraction and remaining aliquots were stored at −20 °C or −80 °C for long term storage.

DNA extractions from saliva were performed by Australian Phenomics Facility (APF) on QIAsymphony SP using QIAsymphony DSP DNA Midi Kit (QIAGEN). In brief, tubes were incubated at 56 °C for 1 h followed by addition of 2 μl of RNAse A (100 mg ml−1, QIAGEN) to 1 ml of saliva sample and incubated at room temperature for 5 min. RNAse activity was suppressed by incubation at 50 °C for 40 min. A custom protocol on QIAsymphony SP, specifically developed for 1 ml of saliva sample was then used to run the DNA extraction process on the instrument. DNA was eluted in 100 μl TE buffer.

Fresh blood collection and DNA extraction

We collected fresh blood where possible from consenting individuals in BD Vacutainer EDTA tubes (lavender caps, BD). Blood tubes were immediately placed on ice after the collection and shipped to the NCIG laboratory on dry ice. Blood samples were stored at −80 °C until required. DNA extraction was performed using FlexiGene DNA Kit (QIAGEN) according to the manufacturer’s protocol. In brief, frozen blood samples were thawed in 37 °C water bath and mixed with the lysis buffer. A cell pellet was then collected by centrifugation at 2,000g for 5 min and supernatant was discarded. Cell pellet was mixed with denaturation buffer and protease enzyme followed by incubation at 65 °C in water bath for 10 min. DNA was precipitated using isopropanol and centrifuged at 2,000g for 3 min. DNA pellet was then washed with 70% ethanol and pelleted at 2,000g for 3 min. Finally, supernatant was discarded and the DNA pellet was air dried. Dry DNA pellet was resuspended in 1 ml of hydration buffer (10 mM Tris-Cl) and incubated at 65 °C for 1 h for dissolution.

For non-Indigenous samples, HMW DNA was previously extracted from blood, using Qiagen DNeasy Blood and Tissue Kit, as per manufacturer’s instructions, and stored at −80 °C.

Whole-genome ONT sequencing

HMW DNA samples were transferred to the Garvan Institute Sequencing Platform for long-read sequencing analysis on Oxford Nanopore Technologies (ONT) instruments. DNA quantity was measured using a Qubit (Thermo Fisher Scientific), purity on a NanoDrop (Thermo Fisher Scientific) and fragment-size distribution on a TapeStation (Agilent). Prior to ONT library preparations, DNA was sheared to ~15–20 kb fragment size using Covaris G-tubes. No shearing was performed on samples where the starting fragment distribution peaked at or below ~25 kb. Sequencing libraries were prepared from ~1–2ug of DNA, using native library preparation kits (either SQK-LSK110 or SQK-LSK114), according to the manufacturer’s instructions. Each library was loaded onto a PromethION flow cell (R9.4.1 for SQK-LSK110 libraries, R10.4.1 for SQK-LSK114 libraries) and sequenced on an ONT PromethION P48 device. Samples were run for a maximum duration of 72 h, with 1–3 nuclease flushes and reloads performed during the run, where necessary to maximize sequencing yield.

ONT data processing

Raw ONT sequencing data were converted from FAST5 to the more compact BLOW5 format[43] in real-time on the PromethION during each sequencing run using slow5tools[44] (v.0.3.0). BLOW5 data were transferred to the Australian National Computational Infrastructure (NCI) high-performance computing environment before further processing. Data were base-called with Guppy (v.6.0.1), using the Buttery-eel wrapper for BLOW5 input[45], with the high-accuracy model and reads with mean quality <7 were excluded from further analysis.

Alignment to reference genome

To evaluate the use of hg38 and T2T-chm13 reference genomes, ONT libraries generated in our study for the HG001 and HG002 reference samples and matched Illumina libraries from the GIAB consortium were mapped against each reference genome. The short-read data were mapped using bwa-mem2 (v.2.2.1), with -Y optional parameter, and the long-read data were mapped using minimap2[46] (v.2.22) with the following optional parameters: -x map-ont -a–secondary=no–MD. The alignment of each individual library to either hg38 or T2T-chm13 was made in a sex-specific manner with an XY reference for genotypically male individuals and an XO reference for genotypically female individuals. After selecting T2T-chm13 as our central reference genome, all other ONT libraries in the cohort were also mapped to this reference using minimap2, as just described.

Detection of non-human reads

To assess the impact of microbial contamination in our sequenced libraries, we first used Centrifuge[47] to identify and classify all non-human reads. We then measured the rate of alignment for these reads to the chm13 reference genome within our standard workflow (see above). We found negligible erroneous alignment of non-human reads to human chromosomes, thereby mitigating the risk of microbial reads causing detection of erroneous variants (see Extended Data Fig. 1a,b).

Detection of structural variation

Detection of large indels (20–49 bp) and SVs (≥50 bp) on HG001 and HG002 Illumina mapped libraries was performed using smoove (v.0.2.6) with default parameters. Variant detection with ONT mapped libraries was performed on each individual sample using CuteSV[24] (v.1.0.13) with the following optional parameters: --max_cluster_bias_INS 100 --diff_ratio_merging_INS 0.3 --max_cluster_bias_DEL 100 --diff_ratio_merging_DEL 0.3 --report_readid --min_support 5 --min_size 20 --max_size 1000000 --genotype. Deletions with <20% of supporting reads for the variant sequence were excluded and insertions with <5% of supporting reads were also excluded. Individual callsets were then merged into a unified joint-call catalogue using Jasmine[27] with the following optional parameters: --min_support = 1 --mark_specific spec_reads= 7 spec_len=20 --pre_normalize --output_genotypes --allow_intrasample–clique_merging --dup_to_ins --normalize_type --run_iris iris_args=min_ins_length=20, --rerunracon, --keep_long_variants. Variants in the joint non-redundant callset were filtered to exclude events with weak evidence (QUAL ≤ 5).

Benchmarking against HGSVC_v4 HG002

We assessed our SV detection strategy using ONT data from the HG002 reference individual, using data from both LSK110/R9.4.1 and LSK114/R10.4.1 ONT chemistry. To determine the average coverage of each library, we computed the total number of bases across all aligned reads and divided it by the genome size. Subsequently, we conducted downsampling by randomly selecting reads until reaching the target coverage. This downsampling process was repeated iteratively to achieve coverages ranging from 30X down to 5X for both datasets.

To evaluate SV detection accuracy, we compared our results to the HGSVC (Freeze 4) annotation for HG002, which served as the ‘truth set’. SVs were deemed true positives (TP) if they matched in type, exhibited a minimum of 50% reciprocal overlap, and had breakpoints within a 200 bp range of each other. SVs in our dataset that did not meet these criteria were classified as false positives (FP), while SVs present in the truth set without a corresponding match were considered false negatives (FN). We used precision and recall metrics to assess the performance of our SV detection method at different coverage levels. Precision was calculated as TP/(TP + FP). Recall was calculated as TP/(TP + FN). Results are presented in Extended Data Fig. 2a.

Structural variation repeat classification

Indels and SVs were classified according to repeat type using custom analysis methods. We first created an extended local allele sequence for each variant, which was 5× the size of the variant itself. For each insertion, we created an extended ALT allele by extracting reference sequence from the T2T-chm13 genome immediately upstream (2× variant size) and downstream (2× variant size) of the variant site, then concatenating these in appropriate order with the consensus insertion sequence that was retrieved from the Jasmine VCF. For each deletion, we created an extended REF allele by extending the variant position in either direction (by 2× variant size) and extracting reference sequence from the T2T-chm13 genome.

Each extended allele, which captures the variant in its local sequence context, was then scanned for tandem repeats using Tandem Repeat Finder[48] (trf409.linux64) with input parameters recommended by the developers (2 7 7 80 10 50 500). Annotated tandem repeats were parsed by their period: 1 bp, homopolymer (HOMO); 2–12 bp, STR; >12 bp, TR. Any overlapping annotations of the same type were merged. We then calculated the extent to which the variant site (that is, central 20% of the local sequence allele) was covered by repeats of each type; if ≥75% of the variant was covered by repeats of a single type, the variant was classified accordingly as either HOMO, STR or TR.

Each extended allele was then scanned for interspersed mobile elements using RepeatMasker (4.1.2-p1) with the following input parameters: -species human -gff -s -norna -nolow. Annotated interspersed repeats were parsed into different types (SINE, LINE, DNA transposon, LTR, retroposon or other), based on RepeatMasker classifications, and labelled as ‘complete’ (≥75%) or ‘fragment’ (<75%) based on the fraction of the canonical sequence element that was present. We then calculated the extent to which the variant site within its local allele sequence was covered by interspersed repeats; if ≥75% of the variant was covered by an element or elements of a single type, the variant was classified accordingly as either: SINE, LINE, DNA transposon, LTR, retroposon or other. If the variant itself covered at least ≥75% of one or more complete annotated elements, the variant was labelled as a ‘complete’ transposition event. If not, it was labelled as a mobile element ‘fragment’, which are mostly small SVs contained within larger interspersed elements. Variants that were not classified with either a tandem or interspersed repeat label, were considered ‘non-repetitive’.

We detected 6,947 (2.3%) homopolymeric deletions–insertions, although we note these are likely to be enriched for technical errors, based on known ONT sequencing error profiles[49], and they were therefore excluded from subsequent analyses.

Comparison to annotations

To assess the novelty of our SV catalogue, we compared SVs to three reference datasets: (1) the gnomAD (v.2.1) SV database (http://ncbi.nlm.nih.gov/sites/dbvarapp/studies/nstd166/); (2) an SV callset from population-scale ONT sequencing of Icelanders published recently by deCODE genetics (http://github.com/DecodeGenetics/LRS_SV_sets); (3) and the HGSVC (freeze 4; http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/HGSVC2). To ensure comparability, SV coordinates were first converted from T2T-chm13 to the hg38 reference genome using the LiftOver utility from UCSC. Successfully lifted SVs were then intersected with gnomAD, deCODE and HGSVC annotations, separately. To account for variability in methods of SV detection between the different studies, as well as potential confounding effects during LiftOver, we allowed for some discrepancy in the placement of breakpoints; SVs were classified as having high (>80%), moderate (50–80%), low (<50%) or no (0%) reciprocal overlap to an annotated SV in gnomAD, deCODE and/or HGSVC. We considered SVs with high and moderate reciprocal overlap as ‘annotated’ and SVs with low or no reciprocal overlap as not ‘unannotated’, and therefore potentially novel.

Telomere distance

The distance of each variant to the nearest telomere was calculated and based on that distance variants were binned into 500 kb fixed windows. The number of variants in each bin was counted and averaged across all chromosomes to assess the density of variant distribution across a generic chromosome. There was a higher density of variants within the 5 Mb of the telomere region, including acrocentric and metacentric chromosomes. To investigate the types of variants driving this effect, we parsed the variant counts in the 500-kb bins based on variant type. We then fitted a LOESS curve onto the log-transformed counts of the different bins for each variant type, displaying the results in an exponential scale, to ensure that y values in the regression were all greater than 0. The curves for different variant types showed that tandem repeats were the variants mostly driving the higher density near telomeres.

PCOA and genetic distance

The non-redundant variant callset was converted into a binary matrix, with rows representing variants and columns representing individuals. The presence of a variant in an individual was represented with a 1 and the absence with a 0. We then used the vegdist function from the vegan R package to calculate the dissimilarity between individuals based on their variant composition using Bray–Curtis methodology. Bray–Curtis was chosen for: (1) its ability to distinguish closely related individuals; (2) its robustness to experimental variables (for example, variable coverage) that may result in missing data; and (3) its previously demonstrated usage for analysis of SVs[20].

Subsequently, we performed a PCOA on the dissimilarity matrix using the pcoa function from the ape R package. PCOA1 and PCOA2, representing principal coordinate axes 1 and 2, were plotted for each individual according to their community/group. We calculated the percent of variance explained by PCOA1 and PCOA2 by dividing their corresponding eigenvalues to the total sum of eigenvalues for all axes multiplied by 100.

To calculate pairwise FST, we randomly selected 10,000 SVs with a frequency higher than 10% in our cohort. This data was converted into a matrix, where rows represented individuals, the first column indicated the group (NCIG-P1, NCIG-P2, NCIG-P3, NCIG-P4 and non-NCIG) and subsequent columns represented loci. The pairwise FST values between groups was then calculated using the gene.dist function from the hierfstat R package, employing the Weir and Cockerham method[50] (method = WC84).

Discovery curves

To measure SV diversity, we generated discovery curves, wherein we calculate the number of new non-redundant SVs gained as additional individuals are considered. Starting with a single NCIG individual, the number of non-redundant variants was calculated each time a new individual was added to the analysis, until all 141 individuals were included. The growth rate of the non-redundant set declines as the number of cumulative individuals increases. We then used the values obtained in the discovery curve to generate a log regression model of the number of non-redundant variants as a function of the number of individuals sampled with the lm function from the stats R package. The curves model the level of heterogeneity in a given group, and enable estimation of the number of individuals required to saturate variant discovery. We then generated discovery curves and log regression models by parsing the variants for each community/group (NCIG-P1, NCIG-P2, NCIG-P3, NCIG-P4, non-NCIG and NCIG-P1/NCIG-P2/NCIG-P3 combined), according to geographical distribution (NCIG-only, NCIG-absent, global) and variant type (non-repetitive, tandem repeats and mobile elements).

LOEUF constraint analysis

We binned variants in the non-redundant callset intersecting protein-coding genes based on their LOEUF decile, previously assigned by gnomAD, and which measures intolerance to loss-of-function variation. Genes in the 1st decile have the highest constraint, while genes in the 10th decile are the least constrained. Therefore, if variants (large indels and/or SVs) regularly have deleterious effects on gene function, the expectation would be that genes in the 1st decile would harbour relatively fewer variants than genes in higher deciles, after accounting for gene size. To test this, we calculated the variant density within CDS and non-CDS regions (introns, UTRs and ±2 kb flanking regulatory regions) of all the genes in each LOEUF decile. Density was calculated by counting the number of non-redundant variants intersecting CDS regions of all genes in a given decile, divided by the total size of CDS regions of all genes of that decile. Similarly, we counted the number of variants intersecting non-CDS regions (but not intersecting CDS regions) and divided that by the total size of non-CDS regions of all the genes in a given decile. Variant density was plotted per LOEUF decile for CDS & non-CDS regions, showing clear differences between high and low deciles for both CDS and non-CDS regions.

Analysis of STRs

To explore the landscape of STR variation, we retrieved all joint-called indels and SVs classified above as ‘STR’ variants, which represent expansions (that is, insertions) and contractions (that is, deletions) of local STR elements. For each variant, we recorded the total expansion/contraction size, the period size (2–12 bp) and the STR motif identified by Tandem Repeat Finder (see ‘Structural variation repeat classification’ above), and investigated global frequencies for each of these dimensions. For STR motif frequency analysis, we considered all possible motif representations in both orientations as a single redundant motif (for example, CAG, AGC, GCA, TGC, GCT, CTG are a single redundant triplet). To identify significantly expanded STR sites, we applied the following criteria: (1) STR period size ≥3 bp; (2) local STR element expanded by at least ≥10 repeats; (3) local STR element is expanded by ≥50% of its reference size; and (4) local STR element reference size is <1 kb. This identified 651 STR sites within protein-coding genes that were significantly expanded in at least one individual.

Individual-level, diploid genotyping of STR alleles was used to elucidate full allelic diversity at the 651 STR sites within protein-coding genes that were significantly expanded in at least one individual (see above), as well as 50 known disease-associated STR loci. This was performed using a custom analysis method. In brief, we identified variation within the local region around a given STR site using clair3[51] (v.0.1-r12; for SNVs and 2–20 bp indels) and sniffles2[52] (v.2.0.2; for indels/SVs >20 bp). Sniffles2 was used instead of CuteSV (as above) because it shows better performance at STR sites when guided by the --tandem-repeats input parameter. Variants from clair3 and sniffles2 were incorporated in a haplotype-specific fashion into the local genome sequence using bcftools consensus (v.1.12)[53], and the modified hap1/hap2 sequences were extracted in a ±50-bp window centred on the STR site; these constitute the consensus STR allele sequences for a given individual at a given STR site, with the larger being designated ‘allele_A’ and the shorter ‘allele_B’. Tandem Repeat Finder was used to determine the STR period size, length, motif and other summary statistics for each STR allele. Allele sequences were visualized in sequence bar charts, in which each tile represented a nucleotide (A, C, G and T), using R package ggplot2. To investigate the variability of STRs within the different communities, we calculated the mean and standard deviation of STR lengths for the alleles within each community. We then performed an ANOVA test (P < 0.05) to identify STR sites that were significantly variable between communities. We plotted the standard deviation for each significantly variable site, normalized to range between 0 and 1, as a heatmap and also performed hierarchical clustering using the heatmap.2 function of the gplots R package.

Analysis of large CNVs

We detected large CNVs (>50 kb) in individual libraries using CNVpytor (v.1.3.1) with a 10 kb bin size. To maintain only high-confidence predicted CNVs, we excluded calls with a P value > 10−4. We grouped CNVs with more than 50% reciprocal overlap into merged regions of variable copy number, encompassing all the individual calls contained in that region. We counted both the number of CNVs identified in each individual and the number of individuals with a call within a given CNV region. That way we classified CNV regions according to the number of different individuals with a call within that CNV region (singleton: 1 individual; polymorphic: >2 and <50% of individuals; major: ≥50% of individuals and <all individuals) and also range depending on its distribution across NCIG and non-NCIG groups (NCIG-only: only found among NCIG individuals; NCIG-absent: only found among non-NCIG individuals; global: found among both NCIG and non-NCIG individuals). To identify annotated protein-coding genes within CNV regions, we used bedtools intersect with a requirement for complete gene containment within a given region (parameter -f 1.0). Read-depth visualization for multiple samples at a specific CNV region was achieved by extracting values from bigwig files and normalizing based on library size.

Data analysis

All data manipulation and visualization, as well as plotting was performed in R (v.4.0.0).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

The following publicly accessible datasets were used in this study: (1) the gnomAD (v.2.1) SV database: http://ncbi.nlm.nih.gov/sites/dbvarapp/studies/nstd166/; (2) deCODE genetics SV callset: http://github.com/DecodeGenetics/LRS_SV_sets; and (3) HGSVC (freeze 4): http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/HGSVC2. The following reference genomes were used in this study: T2T-chm13 (v.2.0): https://github.com/marbl/CHM13 and Hg38 (GRCh38.p13): https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_000001405.39/. All raw sequencing data, processed output files and associated metadata are permanently stored on Australia’s National Computational Infrastructure (NCI) under the control of the Collection Access and Research Advisory Committee (CARAC) appointed and overseen by the NCIG Indigenous-majority governance board. Requests for access by external researchers will be considered by CARAC and governed by the NCIG Board. Data access requests from external researchers may be granted when the board is satisfied that core principles of Indigenous engagement are observed within the proposed research. At the heart of this is the requirement that the proposed research will be of benefit to Australian Indigenous peoples and is identified as important by the communities whose data is involved. Further information can be found within the NCIG governance framework (https://ncig.anu.edu.au/files/NCIG-Governance-Framework.pdf). Data access requests should be directed to jcsmr.ncig@anu.edu.au.

Code availability

All original code has been deposited at Zenodo and is publicly available from https://doi.org/10.5281/zenodo.10020534.

References

1. 1000 Genomes Project Consortium. A global reference for human genetic variation. Nature 526, 68–74 (2015).

2. Karczewski, K. J. et al. The mutational constraint spectrum quantified from variation in 141,456 humans. Nature 581, 434–443 (2020).

3. Liao, W.-W. et al. A draft human pangenome reference. Nature 617, 312–324 (2023).

4. De Coster, W., Weissensteiner, M. H. & Sedlazeck, F. J. Towards population-scale long-read sequencing. Nat. Rev. Genet. 22, 572–587 (2021).

5. Chintalaphani, S. R., Pineda, S. S., Deveson, I. W. & Kumar, K. R. An update on the neurological short tandem repeat expansion disorders and the emergence of long-read sequencing diagnostics. Acta Neuropathol. Commun. 9, 98 (2021).

6. Clarkson, C. et al. The archaeology, chronology and stratigraphy of Madjedbebe (Malakunanja II): a site in northern Australia with early occupation. J. Hum. Evol. 83, 46–64 (2015).

7. O'Connell, J. F. & Allen, J. The process, biotic impact, and global implications of the human colonization of Sahul about 47,000 years ago. J. Archaeol. Sci. 56, 73–84 (2015).

8. Malaspinas, A.-S. et al. A genomic history of Aboriginal Australia. Nature 538, 207–214 (2016).

9. Clarkson, C. et al. Human occupation of northern Australia by 65,000 years ago. Nature 547, 306–310 (2017).

10. Arthur, W. S. & Morphy, F. Macquarie Atlas of Indigenous Australia: Culture and Society Through Space and Time (Macquarie Library, Macquarie Univ., 2005).

11. Bowern, C. & Atkinson, Q. Computational phylogenetics and the internal structure of Pama-Nyungan. Language 88, 817–845 (2012).

12. Fatumo, S. et al. A roadmap to increase diversity in genomic studies. Nat. Med. 28, 243–250 (2022).

13. Easteal, S. et al. Equitable expanded carrier screening needs Indigenous clinical and population genomic data. Am. J. Hum. Genet. 107, 175–182 (2020).

14. D’Angelo, C. S. et al. Barriers and considerations for diagnosing rare diseases in Indigenous populations. Front. Pediatr. 8, 579924 (2020).

15. Lewis, D. Australian biobank repatriates hundreds of ‘legacy’ Indigenous blood samples. Nature 577, 11–12 (2020).

16. Hermes, A. et al. Beyond platitudes: a qualitative study of Australian Aboriginal people’s perspectives on biobanking. Intern. Med. J. 51, 1426–1432 (2021).

17. Nurk, S. et al. The complete sequence of a human genome. Science 376, 44–53 (2022).

18. Logsdon, G. A., Vollger, M. R. & Eichler, E. E. Long-read human genome sequencing and its applications. Nat. Rev. Genet. 21, 597–614 (2020).

19. Collins, R. L. et al. A structural variation reference for medical and population genetics. Nature 581, 444–451 (2020).

20. Sudmant, P. H. et al. An integrated map of structural variation in 2,504 human genomes. Nature 526, 75–81 (2015).

21. Zook, J. M. et al. An open resource for accurately benchmarking small variant and reference calls. Nat. Biotechnol. 37, 561–566 (2019).

22. Harvey, W. T. et al. Whole-genome long-read sequencing downsampling and its effect on variant calling precision and recall. Preprint at bioRxiv https://doi.org/10.1101/2023.05.04.539448 (2023).

23. Kirby, A. et al. Mutations causing medullary cystic kidney disease type 1 lie in a large VNTR in MUC1 missed by massively parallel sequencing. Nat. Genet. 45, 299–303 (2013).

24. Jiang, T., Liu, S., Cao, S. & Wang, Y. Structural variant detection from long-read sequencing data with cuteSV. Methods Mol. Biol. 2493, 137–151 (2022).

25. Beyter, D. et al. Long-read sequencing of 3,622 Icelanders provides insight into the role of structural variants in human diseases and other traits. Nat. Genet. 53, 779–786 (2021).

26. Chaisson, M. J. P. et al. Multi-platform discovery of haplotype-resolved structural variation in human genomes. Nat. Commun. 10, 1784 (2019).

27. Kirsche, M. et al. Jasmine: population-scale structural variant comparison and analysis. Nat Meth 20, 408–417 (2023).

28. Suvakov, M., Panda, A., Diesh, C., Holmes, I. & Abyzov, A. CNVpytor: a tool for copy number variation detection and analysis from read depth and allele imbalance in whole-genome sequencing. Gigascience 10, giab074 (2021).

29. Audano, P. A. et al. Characterizing the major structural variant alleles of the human genome. Cell 176, 663–675 (2019).

30. Tang, W., Mun, S., Joshi, A., Han, K. & Liang, P. Mobile elements contribute to the uniqueness of human genome with 15,000 human-specific insertions and 14 Mbp sequence increase. DNA Res. 25, 521–533 (2018).

31. Paulson, H. Machado–Joseph disease/spinocerebellar ataxia type 3. Handb. Clin. Neurol. 103, 437–449 (2012).

32. Carr, J. J. et al. ‘Staying strong on the inside and outside’ to keep walking and moving around: perspectives from Aboriginal people with Machado Joseph disease and their families from the Groote Eylandt Archipelago, Australia. PLoS ONE 14, e0212953 (2019).

33. Elsum, I. et al. A community-based co-designed genetic health service model for Aboriginal Australians. PLoS ONE 15, e0239765 (2020).

34. Stevanovski, I. et al. Comprehensive genetic diagnosis of tandem repeat expansion disorders with programmable targeted nanopore sequencing. Sci. Adv. 8, eabm5386 (2022).

35. Wright, J. L. et al. Ancient nuclear genomes enable repatriation of Indigenous human remains. Sci. Adv. 4, eaau5064 (2018).

36. Rasmussen, M. et al. An Aboriginal Australian genome reveals separate human dispersals into Asia. Science 334, 94–98 (2011).

37. Bergström, A. et al. Deep roots for Aboriginal Australian Y chromosomes. Curr. Biol. 26, 809–813 (2016).

38. Nagle, N. et al. Mitochondrial DNA diversity of present-day Aboriginal Australians and implications for human evolution in Oceania. J. Hum. Genet. 62, 343–353 (2017).

39. Tobler, R. et al. Aboriginal mitogenomes reveal 50,000 years of regionalism in Australia. Nature 544, 180–184 (2017).

40. Wu, Z. et al. Structural variants in the Chinese population and their impact on phenotypes, diseases and population adaptation. Nat. Commun. 12, 6501 (2021).

41. Otsuki, A. et al. Construction of a trio-based structural variation panel utilizing activated T lymphocytes and long-read sequencing technology. Commun Biol 5, 991 (2022).

42. Beecroft, S. J. et al. A Māori specific RFC1 pathogenic repeat configuration in CANVAS, likely due to a founder allele. Brain 143, 2673–2680 (2020).

43. Gamaarachchi, H. et al. Fast nanopore sequencing data analysis with SLOW5. Nat. Biotechnol. 40, 1026–1029 (2022).

44. Samarakoon, H. et al. Flexible and efficient handling of nanopore sequencing signal data with slow5tools. Genome Biol. 24, 69 (2023).

45. Samarakoon, H., Ferguson, J. M., Gamaarachchi, H. & Deveson, I. W. Accelerated nanopore basecalling with SLOW5 data format. Bioinformatics 39, btad352 (2023).

46. Li, H. Minimap2: pairwise alignment for nucleotide sequences. Bioinformatics 34, 3094–3100 (2018).

47. Kim, D., Song, L., Breitwieser, F. P. & Salzberg, S. L. Centrifuge: rapid and sensitive classification of metagenomic sequences. Genome Res. 26, 1721–1729 (2016).

48. Benson, G. Tandem repeats finder: a program to analyze DNA sequences. Nucleic Acids Res. 27, 573–580 (1999).

49. Wang, Y., Zhao, Y., Bollas, A., Wang, Y. & Au, K. F. Nanopore sequencing technology, bioinformatics and applications. Nat. Biotechnol. 39, 1348–1365 (2021).

50. Weir, B. S. & Cockerham, C. C. Estimating F-statistics for the analysis of population structure. Evolution 38, 1358–1370 (1984).

51. Zheng, Z. et al. Symphonizing pileup and full-alignment for deep learning-based long-read variant calling. Nat. Comput. Sci. 2, 797–803 (2022).

52. Sedlazeck, F. J. et al. Accurate detection of complex structural variations using single-molecule sequencing. Nat. Methods 15, 461–468 (2018).

53. Danecek, P. et al. Twelve years of SAMtools and BCFtools. Gigascience 10, giab008 (2021).

Acknowledgements

We acknowledge the Aboriginal and Torres Strait Islander peoples as the first peoples and traditional custodians of the lands and waters where we meet, live, learn and work. We celebrate the rich diversity of Aboriginal and Torres Strait Islander cultures and the ongoing leadership of our First Nations’ peoples and communities who have paved the way. We pay our respects to ancestors of this country, the legacy of elders, the knowledge holders, and leaders of the past, present, and future. This work was conducted primarily on land traditionally owned by the Ngunnawal and Ngambri peoples and the Gadigal people of the Eora Nation. We are indebted to the individuals and communities who participated in this research and the NCIG Governance Board who guided this work in a culturally appropriate manner. We acknowledge the following Community Organizations and individuals: Yarrabah Shire Council, R. Andrews, E. Fourmile and P. Burns; Tiwi Land Council Board members; Yalu Aboriginal Corporation (Galiwin’ku), R. Wunungmurra, E. Djotja, R. Gundjarrangbuy; Titjikala Shire Council and Titjikala Health services. We thank our colleagues A. McCarthy, W. Hoy, S. Foote, J. Matthews, R. Thomson, D. MacArthur, J. Yuan, T. Amos and J. Craig for their various contributions to the project. This project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government and the Australian National University (ANU). We acknowledge the following facilities that were used during this study: the Garvan Sequencing Platform (GSP) and the Australian Phenomics Facility (APF). We acknowledge the following funding sources: Medical Research Futures Fund (MRFF) grants 2016008, 1173594, 2023126 and 2016124, and National Health and Medical Research Council (NHMRC) grants 2011277 and 2021172.

Author information

Authors and Affiliations

1. Genomics and Inherited Disease Program, Garvan Institute of Medical Research, Sydney, New South Wales, Australia

Andre L. M. Reis, Melissa Rapadas, Jillian M. Hammond, Hasindu Gamaarachchi, Igor Stevanovski, Meutia Ayuputeri Kumaheri, Sanjog R. Chintalaphani, Owen M. Siggs & Ira W. Deveson

2. Centre for Population Genomics, Garvan Institute of Medical Research and Murdoch Children’s Research Institute, Darlinghurst, New South Wales, Australia

Andre L. M. Reis, Melissa Rapadas, Jillian M. Hammond, Hasindu Gamaarachchi, Igor Stevanovski, Meutia Ayuputeri Kumaheri, Sanjog R. Chintalaphani, Owen M. Siggs & Ira W. Deveson

3. Faculty of Medicine, University of New South Wales, Sydney, New South Wales, Australia

Andre L. M. Reis, Sanjog R. Chintalaphani & Ira W. Deveson

4. School of Computer Science and Engineering, University of New South Wales, Sydney, New South Wales, Australia

Hasindu Gamaarachchi

5. National Centre for Indigenous Genomics, John Curtin School of Medical Research, Australian National University, Canberra, Australian Capital Territory, Australia

Duminda S. B. Dissanayake, Bastien Llamas, Alex Brown, Graham J. Mann, Brendan J. McMorran, Simon Easteal, Azure Hermes, Graham Mann & Hardip R. Patel

6. Institute for Applied Ecology, University of Canberra, Canberra, Australian Capital Territory, Australia

Duminda S. B. Dissanayake

7. Department of Ophthalmology, Flinders University, Bedford Park, South Australia, Australia

Owen M. Siggs

8. Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania, Australia

Alex W. Hewitt

9. Australian Centre for Ancient DNA, School of Biological Sciences and Environment Institute, University of Adelaide, Adelaide, South Australia, Australia

Bastien Llamas

10. ARC Centre of Excellence for Australian Biodiversity and Heritage, University of Adelaide, Adelaide, South Australia, Australia

Bastien Llamas

11. Indigenous Genomics, Telethon Kids Institute, Adelaide, South Australia, Australia

Bastien Llamas & Alex Brown

12. Telethon Kids Institute and Division of Paediatrics, Faculty of Health and Medical Sciences, University of Western Australia, Perth, Western Australia, Australia

Gareth Baynam & Glen Pearson

13. Genetic Services of Western Australia, Western Australian Department of Health, Perth, Western Australia, Australia

Gareth Baynam

14. Western Australian Register of Developmental Anomalies, Western Australian Department of Health, Perth, Western Australia, Australia

Gareth Baynam

15. Immunology Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia

Misty R. Jenkins

16. Molly Wardaguga Research Centre, Faculty of Health, Charles Darwin University, Brisbane City, Queensland, Australia

Yvette Roe

17. Mater Research Institute and School of Nursing and Midwifery, University of Queensland, South Brisbane, Queensland, Australia

Yvette Roe

18. The Lowitja Institute, Melbourne, Victoria, Australia

Janine Mohamed

19. Office of the Registrar of Indigenous Corporations, Canberra, Australian Capital Territory, Australia

Ben Murray

20. School of Media and Communication, RMIT University, Melbourne, Victoria, Australia

Lyndon Ormond-Parker

21. Centre of Excellence for Automated Decision-Making and Society, RMIT University, Melbourne, Victoria, Australia

Lyndon Ormond-Parker

22. Commonwealth Scientific and Industrial Research Organisation, Canberra, Australian Capital Territory, Australia

Erica Kneipp

23. The Australian National University, Canberra, Australian Capital Territory, Australia

Keith Nugent

Consortia

The National Centre for Indigenous Genomics

- Glen Pearson

- , Yvette Roe

- , Janine Mohamed

- , Ben Murray

- , Lyndon Ormond-Parker

- , Gareth Baynam

- , Erica Kneipp

- , Keith Nugent

- & Graham Mann

Contributions

I.W.D. and H.R.P. conceived the study, with the support of G.J.M., A.H., B.J.M., M.R.J. and S.E. A.B., A.H., M.R.J. and the NCIG provided Indigenous leadership and oversight to the project. The NCIG provided access to DNA samples from Indigenous communities. O.M.S. and A.W.H. provided access to non-Indigenous comparison data. M.R., J.M.H., I.S., M.A.K. and D.S.B.D. processed samples and performed sequencing experiments. A.L.M.R., H.G., S.R.C., H.R.P. and I.W.D. performed bioinformatics analysis. A.L.M.R. and I.W.D. generated the Figures. A.L.M.R., H.R.P and I.W.D. wrote the manuscript, with input from all co-authors.

Corresponding authors

Correspondence to Hardip R. Patel or Ira W. Deveson.

Ethics declarations

Competing interests

This project receives partial in-kind support from Oxford Nanopore Technologies (ONT) under an ongoing collaboration agreement. A.L.M.R., J.M.H., H.G., H.R.P. and I.W.D. have previously received travel and accommodation expenses from ONT to speak at conferences. H.G. and I.W.D. have paid consultant roles with Sequin Pty Ltd. O.M.S. and A.W.H. hold equity in Seonix Pty Ltd. The authors declare no other competing interests.

Peer review

Peer review information

Nature thanks Phillip Wilcox and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer review reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Genomic library characteristics across different groups.

(a) Barchart shows the proportion of non-human reads in sequencing libraries derived from blood or saliva samples. (b) Barchart shows the proportion of mapped (green) and unmapped (orange) non-human reads in sequencing libraries derived from blood or saliva samples. (c) Boxplot shows the average depth of coverage per individual grouped by their communities (NCIGP1 = pink, P2 = purple, P3 = blue, P4 = green & non-NCIG = orange). The horizontal dashed line indicates the average coverage across all libraries in the cohort. (d) Boxplot shows the N50 distribution of individual libraries in the different communities. The horizontal dashed line indicates the average N50 across all libraries in the cohort. (e) Boxplot shows the distribution of DNA Integrity Number (DIN), which indicates the level of fragmentation of a genomic DNA sample, for individual libraries across the different communities. (f) Boxplot shows the distribution of the number of high-quality (PASS=orange) and low-quality (q5=green) structural variants (SVs) per individual grouped by community after quality filtering (Quality ≥ 5). The horizontal dashed lines indicate the average number of high-quality (top) and low-quality (bottom) SVs across all libraries in the cohort. A total of n = 141 individuals (NCIGP1 = 41, NCIGP2 = 32, NCIGP3 = 9, NCIGP4 = 39 and non-NCIG = 20) were examined from independent sequencing experiments in figures c-f. In the boxplots, the middle line is the median, the box represents the interquartile range (IQR), the whiskers extend 1.5 times the IQR from the hinge, and any data points beyond the whiskers are shown individually.

Extended Data Fig. 2 Structural variation detection performance and genomic alignments for HG002 samples.

(a) Line plots show precision and recall of SV calls detected with cuteSV for HG002 samples sequenced with LSK110/R9.4.1 or LSK114/R10.4.1 ONT chemistry and subsampled to different coverages compared against HGSVCv4 reference calls for HG002 (taken as a ‘truth set’). (b) Genome browser views show comparison of short-read and long-read alignments to either the hg38 or chm13-T2T reference genomes at MUC1, an example of a repetitive medically relevant gene. Both datasets are from the HG002 reference sample. The gene contains a large tandem repeat region that is best resolved by alignment of long-reads to chm13-T2T.

Extended Data Fig. 3 Copy number variation analysis across different groups.

(a) Bar chart shows the number of large CNVs (> 50 kb) identified in individuals from each group, broken down by type: deletion (red) and duplication (blue). (b) Bar chart shows the cumulative size of CNVs identified in individuals from each group. The horizontal dashed line indicates the average cumulative CNV size across the entire cohort. (c) Histograms show the size distribution of unique CNV regions (> 50 kb) containing deletions (red) and duplications (blue). The vertical dashed lines indicate the average size for deletions and duplications, respectively. (d) Genome browser views show coverage tracks for 2 individuals from NCIG-P2 (purple) and 1 non-NCIG individual (orange) across chromosomes 11 and 2 of chm13-T2T. In the first panel, the non-NCIG individual has the longest deletion identified, which is indicated by the blue segment and visible in the coverage track for that individual, but missing in the other 2 NCIG-P2 individuals. In the second panel, the 2 NCIG-P2 individuals have the largest duplication identified, also indicated by the blue segment and visible in the respective coverage tracks, but missing in the non-NCIG individual. Centromeric repeats are labeled and represented as red segments.

Extended Data Fig. 4 Genomic distribution and classification of structural variants.

(a) Line plots show LOESS curves of the number of large indels (> 20 & <50 bp) and structural variants (≥ 50 bp) per 500 Kb fixed window relative to the distance to the nearest telomere, parsed by variant type (non-repetitive = teal, short tandem repeat = red, tandem repeat = blue & mobile element = purple). (b) Dot plots show the number of structural variants per 500 Kb fixed window, relative to the distance of the window to the nearest telomere, for acrocentric and metacentric autosomes. The vertical dashed lines indicate a distance of 5 MB from the telomere. (c) Bar chart shows the proportion of SVs that could be lifted (green) or not (orange) from T2T-chm13 to hg38. (d) Bar chart shows the proportion of SVs classified according to reciprocal overlap as high (>80%; pink), moderate (50-80%; brown), low (<50%; beige) or no overlap (purple) in comparison to reference databases (gnomAD, deCODE & HGSVCv4; see Fig. 2e), parsed by SV type.

Extended Data Fig. 5 Distribution and sharedness of large indels and structural variants across the cohort.

(a) Bar chart shows the number of large indels (20-49 bp) identified in individuals from each group (NCIGP1, P2, P3, P4 & non-NCIG), broken down by type: non-repeat (teal) and tandem repeat (STR = red & TR = blue). (b) Bar plot shows the number of non-redundant structural variants identified within a given number of individuals in the cohort (degree of sharedness). Variants were classified as private (1 individual), polymorphic (> 2 & <50% of individuals), major (≥ 50% of individuals & <all individuals) and shared (all individuals). (c) Bar chart shows the proportion of different variant types (same colour scheme as a, in addition to homopolymers = light purple) for large indels identified within a given number of individuals in the cohort (degree of sharedness).

Extended Data Fig. 6 Population distribution and annotation of large indels and structural variants.

(a) Bar chart shows the proportion of large indels in each individual that were only found in NCIG individuals (NCIG-only = purple), only found in non-NCIG individuals (NCIG-absent = green) and found in both NCIG & non-NCIG individuals (Global = light blue). (b) Bar chart shows the proportion of NCIG-only, NCIG-absent and Global large indels (same colour scheme as Fig. 3a) for all the variants identified within a given number of individuals in the cohort (degree of sharedness). (c) Proportion of NCIG-only, NCIG-absent or Global SVs in each individual that were previously annotated in gnomAD, deCODE or HGSVCv4 SV catalogs (see Fig. 2e). Accompanying boxplot shows the distribution of ratios between the proportion of SVs ‘unannotated’ (high & moderate overlap) to the proportion of SVs ‘annotated’ (low and no overlap) against at least one of the databases. A total of n = 141 individuals (NCIG-only = 121, NCIG-absent = 20 and Global = all individuals) were examined from independent sequencing experiments in figure c. In the boxplot, the middle line is the median, the box represents the interquartile range (IQR), the whiskers extend 1.5 times the IQR from the hinge, and any data points beyond the whiskers are shown individually.

Extended Data Fig. 7 Distribution and diversity of genomic variation across different indigenous communities.

(a) Matrix shows pairwise FST calculated using a random set of 10,000 SV loci found among NCIG communities and the non-NCIG group. In the accompanying scale, the intensity of red increases with the FST values. (b) Upset plot shows the distribution of NCIG-only large indels (20-49 bp) shared among the four indigenous communities (NCIGP1, P2, P3 & P4). Variants were classified as private (n = 1 individual; blue), community-specific (n > 1 individual in 1 community; yellow), widespread (n > 1 individual in more than 1 community; red) or shared (n > 1 individual in all 4 communities; green) according to the number of communities in which they were identified. (c) Upset plot shows the distribution of NCIG SNVs shared among the four indigenous communities (NCIGP1, P2, P3 & P4). Community specific variants were highlighted in yellow. (d) Upset plot shows the comparison of NCIG SNVs against the Simon Genome Diversity Project (SGDP), gnomAD and dbSNP154. NCIG specific SNVs were highlighted in green.

Extended Data Fig. 8 Genomic variation distribution and sampling dynamics across the cohort.

(a) Proportion of different SV types for NCIG-only variants classified as private, community-specific, widespread or shared. Types are non-repetitive (teal), tandem repeat (STR = red & TR = blue) and mobile element (fragment = light purple & complete = dark purple). (b) Log regression models predicting the number of non-redundant SVs identified, given the number of individuals sampled. The models are broken down by community (left panel), by geographical distribution (centre panel) and SV type (NCIG individuals; right panel). (c) Bar chart shows a discovery curve, in which starting with a single NCIG individual, the number of new non-redundant large indels is counted by iteratively adding the unique calls from additional NCIG individuals. Indels shared among all previously added samples are shown as green portions of each bar. The growth rate of the nonredundant set declines as the number of samples increases. (d) Log regression model showing the predicted number of non-redundant large indels identified given the number of individuals sampled. The model was broken down by variant type (Non-repetitive = teal, Tandem repeats = red). (e) Proportion of private, community-specific, widespread & shared NCIG-only variants among individuals, grouped by community. A total of n = 141 individuals (NCIGP1 = 41, NCIGP2 = 32, NCIGP3 = 9, NCIGP4 = 39 and non-NCIG = 20) were examined from independent sequencing experiments in figure e. In the boxplot, the middle line is the median, the box represents the interquartile range (IQR), the whiskers extend 1.5 times the IQR from the hinge, and any data points beyond the whiskers are shown individually.

Extended Data Fig. 9 Genomic variation impact on protein-coding genes.

(a) Bar plot shows the number of variants per individual impacting CDS exons of protein-coding genes. The horizontal dashed line indicates the average number of variants in CDS regions across the entire cohort. (b) Bar plots show the cumulative number of whole-genes contained within CNV regions sorted by size in increasing order for deletions and duplications. (c) Bar plots show for each CNV region, the number of different individuals that had a CNV identified within that region and were either intergenic or intersected whole genes. Different CNV regions are classified as singleton (light purple;1 individual), polymorphic (green; > 2 & <50% of individuals) and major (pink; ≥ 50% of individuals & <all individuals). (d) The bar plot shows the number of NCIG-only variants per LOEUF decile parsed by their level of distribution within NCIG communities. Variants were classified as private (n = 1 individual; blue), community-specific (n > 1 individual in 1 community; yellow), widespread (n > 1 individual in more than 1 community; red) or shared (n > 1 individual in all 4 communities; green) according to the number of communities in which they were identified. (e) Genome browser view shows sequencing alignments to ATXN3. A ‘CAG’ STR expansion, known to cause Machado-Joseph Disease (MJD), was identified in one NCIG-P2 individual. ONT reads span the expansion (left panel; purple markers indicate insertions). Illumina short-reads do not span the expansion, and are soft-clipped (right panel).

Extended Data Fig. 10 Analysis of STR expansion characteristics and community-specific variability.

(a) Dot plots show the number of repeat units versus the relative size increase of STR expansions of different period sizes. The horizontal dashed line indicates the minimum relative size increase (0.5) and the vertical dashed line indicates the minimum number of repeat units (10) required for an STR expansion to be further genotyped across all individuals in the cohort. (b) Matrix shows the normalised standard deviation (range 0:1) of allele sizes within each community for all STR sites with expansions in one or more individuals, in which allelic composition between the groups was significantly different. Hierarchical clustering was performed to group the STR sites based on the different patterns of variability between communities.

Supplementary information

Supplementary Table 1

Overview of ONT sequencing datasets generated for this study.

Reporting Summary

Peer Review File

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Reis, A.L.M., Rapadas, M., Hammond, J.M. et al. The landscape of genomic structural variation in Indigenous Australians. Nature 624, 602–610 (2023). https://doi.org/10.1038/s41586-023-06842-7

- Received: 16 January 2023

- Accepted: 07 November 2023

- Published: 13 December 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06842-7

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

- Australian Indigenous genomes are highly diverse and unlike those anywhere else

- Bianca Nogrady

Nature (2023)

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Indigenous Australian genomes show deep structure and rich novel variation

Article Open access 13 December 2023

Matthew Silcocks, Ashley Farlow, … The National Centre for Indigenous Genomics

Whole Genome Sequence, Variant Discovery and Annotation in Mapuche-Huilliche Native South Americans

Article Open access 14 February 2019

Elena A. Vidal, Tomás C. Moyano, … Rodrigo A. Gutiérrez

Structural variants in the Chinese population and their impact on phenotypes, diseases and population adaptation

Article Open access 11 November 2021

Zhikun Wu, Zehang Jiang, … Zhi Xie

Indigenous Australian genomes show deep structure and rich novel variation

- Matthew Silcocks

- Ashley Farlow

- Stephen Leslie

Nature Article Open Access 13 Dec 2023

Indigenous diversity in Australia’s DNA tapestry unlocked for genetic equity

- Katrina G. Claw

- Amber Nashoba

Nature News & Views 13 Dec 2023

Article Open access Published: 13 December 2023

Lung dendritic-cell metabolism underlies susceptibility to viral infection in diabetes

Samuel Philip Nobs,

Aleksandra A. Kolodziejczyk,

Lital Adler,

Nir Horesh,

Christine Botscharnikow,

Ella Herzog,

Gayatree Mohapatra,

Sophia Hejndorf,

Ryan-James Hodgetts,

Igor Spivak,

Lena Schorr,

Leviel Fluhr,

Denise Kviatcovsky,

Anish Zacharia,

Suzanne Njuki,

Dinorah Barasch,

Noa Stettner,

Mally Dori-Bachash,

Alon Harmelin,

Alexander Brandis,

Tevie Mehlman,

Ayelet Erez,

Yiming He,

Sara Ferrini,

Jens Puschhof,

Hagit Shapiro,

Manfred Kopf,

Arieh Moussaieff,

Suhaib K. Abdeen &

…

Eran Elinav

Show authors

Nature volume 624, pages 645–652 (2023)

- 9338 Accesses

- 141 Altmetric

- Metrics details

Abstract

People with diabetes feature a life-risking susceptibility to respiratory viral infection, including influenza and SARS-CoV-2 (ref. [1]), whose mechanism remains unknown. In acquired and genetic mouse models of diabetes, induced with an acute pulmonary viral infection, we demonstrate that hyperglycaemia leads to impaired costimulatory molecule expression, antigen transport and T cell priming in distinct lung dendritic cell (DC) subsets, driving a defective antiviral adaptive immune response, delayed viral clearance and enhanced mortality. Mechanistically, hyperglycaemia induces an altered metabolic DC circuitry characterized by increased glucose-to-acetyl-CoA shunting and downstream histone acetylation, leading to global chromatin alterations. These, in turn, drive impaired expression of key DC effectors including central antigen presentation-related genes. Either glucose-lowering treatment or pharmacological modulation of histone acetylation rescues DC function and antiviral immunity. Collectively, we highlight a hyperglycaemia-driven metabolic-immune axis orchestrating DC dysfunction during pulmonary viral infection and identify metabolic checkpoints that may be therapeutically exploited in mitigating exacerbated disease in infected diabetics.

Similar content being viewed by others

Main

Diabetes mellitus constitutes a major public health burden[2], leading to a variety of complications secondary to a failure to maintain glucose control[3]. One key unexplained phenomenon in both type 1 and 2 diabetes is a markedly increased susceptibility to respiratory infection[1], leading to significantly enhanced morbidity and mortality following infection with viruses such as influenza and other lung pathogens[1]. This phenomenon has recently gained even more pressing importance, with the realization that diabetes constitutes a common comorbidity among patients with severe SARS-CoV-2 (ref. [4]) and promotes markedly worsened pulmonary disease, higher mortality5,6 and risk of serious post-vaccination breakthrough infection[7]. Studies in experimental animal models evaluating the links between respiratory infection and diabetes have been sparse[8], and do not offer a mechanism explaining this pronounced risk[9]. We sought to explore drivers of diabetes-induced susceptibility to pulmonary viral infection.

Diabetes exacerbates viral lung disease

Exact statistical values of all experiments are provided in Supplementary Table 1. We began our investigation by assessing disease severity and immune responses in models of hyperglycaemic mice acutely infected with the H1N1 influenza A/Puerto Rico/8/1934 (PR8) influenza A virus (IAV). Akita mice, mutated in the Ins2 gene, gradually developed systemic hyperglycaemia (Supplementary Information 2a,b) at around 6 weeks of age. Free fluid volume (Supplementary Information 2c) and most other serum electrolytes (Supplementary Information 2d) remained unaltered. Following high-dose IAV infection—and, similar to diabetic humans[1]—Akita mice evidenced markedly enhanced mortality compared with wild-type (WT) littermate controls (Fig. 1a). Increased disease burden in hyperglycaemic mice manifested as significantly elevated viral titres at 10 days post infection (d.p.i.) (Fig. 1a). Infiltration of lymphocytes in IAV-infected WT animals was markedly reduced in infected hyperglycaemic mice (Extended Data Fig. 1a,b), in parallel with impaired viral clearance (Fig. 1a). Collectively, this suggested that a potential defect in antiviral immunity may be linked to a hyperglycaemia-associated susceptibility to influenza infection.

Fig. 1: Diabetes exacerbates lung viral infection.

a,b, WT (n = 27) and Akita (n = 19) mice infected with 200 plaque-forming units (pfu) PR8, log-rank Mantel–Cox test. a, Survival. Inset, lung PR8 RNA 10 d.p.i., WT (n = 16) and Akita (n = 7) mice infected with 50 pfu PR8, two-sided Mann–Whitney U-test. b, Lung NP34-tetramer+CD8+ T cells, two-sided Mann–Whitney U-test. c,d, Mice infected with 50 pfu PR8, treated with insulin (Ins)/phosphate-buffered saline (PBS): WT+PBS (n = 16), WT+Ins (n = 15), Akita+PBS (n = 13), Akita+Ins (n = 15), Kruskal–Wallis test with Dunn’s correction, two pooled experiments. c, Lung PR8 RNA. d, Lung NP34-tetramer+CD8+ T cells. e,f, Mice infected with 200 pfu PR8, administered STZ (n = 18 for e, n = 9 for f) or PBS (n = 30 for e, n = 20 for f). e, Survival, log-rank Mantel–Cox test. Inset, lung PR8 RNA 10 d.p.i., mice infected with 50 pfu PR8, administered STZ (n = 9) or PBS (n = 20), two-sided Mann–Whitney U-test. f, Lung NP34-tetramer+CD8+ T cells, two-sided Mann–Whitney U-test. g,h, Mice infected with 50 pfu PR8, administered STZ or PBS, treated with Ins/PBS: PBS+PBS (n = 27 for g, n = 19 for h), PBS+Ins (n = 19 for g, n = 15 for h), STZ+PBS (n = 10 for g, n = 4 for h), STZ+Ins (n = 17 for g, n = 11 for h), Kruskal–Wallis test with Dunn’s correction. g, PR8 RNA, three pooled experiments. h, Lung NP34-tetramer+CD8+ T cells, two pooled experiments. i,j, Db/Db (n = 9 for i, n = 10 for j) and WT (n = 10 for i, n = 15 for j) mice infected with 200 pfu PR8. i, Survival, log-rank Mantel–Cox test. Inset, lung PR8 RNA 10 d.p.i., WT (n = 20) and Db/Db (n = 10) mice infected with 50 pfu PR8, three pooled experiments, two-sided Mann–Whitney U-test. j, NP34-tetramer+CD8+ T cells, two-sided Mann–Whitney U-test. k, Survival, WT (n = 15) and Akita (n = 18) mice infected with 200 pfu PVM, log-rank Mantel–Cox test. Inset, lung PVM RNA 10 d.p.i., WT (n = 15) and Akita (n = 17) mice infected with 50 pfu PVM, two-sided Mann–Whitney U-test. All P values are indicated in Supplementary Table 1. All data mean+s.e.m. a.u., Arbitrary units.

Pulmonary interferon (IFN)-β (Ifnb1) gene expression (Supplementary Information 2e) and bronchoalveolar lavage (BAL) levels of several IFN-induced proteins (IP-10 and MCP-2; Supplementary Information 2f–g) were significantly elevated in hyperglycaemic compared with normoglycaemic IAV-infected mice, probably reflecting an intact type I IFN response secondary to increased viral titres and tissue damage. By contrast, the adaptive immune response in IAV-infected hyperglycaemic Akita mice was notable for a significant reduction in the number of total lung CD8+ and CD4+ T cells (Extended Data Fig. 1c) and virus-specific CD8+ T cells, using a major histocompatibility complex (MHC) tetramer directed against the immunodominant epitope NP34 (Fig. 1b and Supplementary Information 3a). Furthermore, the frequency of proliferating Ki-67+CD8+ T cells (Extended Data Fig. 1d and Supplementary Information 3b), IFNγ-producing CD8+ T cells (Extended Data Fig. 1e and Supplementary Information 3b,c) and T-bet+CD8+ T cells (Extended Data Fig. 1f) was reduced in hyperglycaemic animals compared with infected non-diabetic controls. A slight increase was observed in the frequency of lung FoxP3+CD4+ T regulatory cells (Treg cells; Extended Data Fig. 1g). In addition, reduced numbers of total and CD95+ pulmonary germinal centre B cells (Extended Data Fig. 1h,i), coupled with significantly lower antiviral antibody titres in BAL (IgG2b and IgM; Extended Data Fig. 1j–m), were noted in infected hyperglycaemic mice compared with normoglycaemic controls. Importantly, glucose normalization in IAV-infected Akita mice using continuous insulin supplementation reduced blood glucose levels (Extended Data Fig. 1n), lowered viral titres (Fig. 1c) and rescued antiviral CD4+ and CD8+ T cells and antibody titres (Fig. 1d and Extended Data Fig. 1o–s). Taken together, these findings suggest that an increased susceptibility to respiratory infection during hyperglycaemia is coupled with a broadly impaired pulmonary antiviral adaptive immunity.

To generalize these findings, we used a model of hyperglycaemia induced by the administration of streptozotocin (STZ), which leads to rapid destruction of pancreatic insulin-producing β cells, driving acute hyperglycaemia (Extended Data Fig. 2a). Similar to the Akita model, infection of STZ-treated mice with IAV was associated with enhanced mortality compared with infected normoglycaemic animals (Fig. 1e) whereas no mortality enhancement was noted in non-infected STZ-treated animals (Extended Data Fig. 2b). Enhanced disease severity in STZ-treated mice was associated with higher type I IFN responses (Extended Data Fig. 2c), elevated viral titres (Fig. 1e), reduced pulmonary immune infiltration (Extended Data Fig. 2d,e) and severely impaired T and B cell antiviral responses, with no effect noted on numbers of lung Treg cells (Fig. 1f and Extended Data Fig. 2f–r). Insulin replenishment to diabetic STZ-treated animals rescued viral clearance and survival (Fig. 1g and Extended Data Fig. 3a) and improved antiviral adaptive immunity (Fig. 1h and Extended Data Fig. 3b–d). In a third, type 2 diabetes model, in which hyperglycaemia spontaneously develops in leptin receptor-deficient (Db/Db) mice (Extended Data Fig. 3e), IAV infection induced lower survival (Fig. 1i), elevated viral titres (Fig. 1i) and impaired adaptive immunity in diabetic mice (Fig. 1j and Extended Data Fig. 3f–j) compared with infected non-diabetic littermate controls. Importantly, and in contrast to the above type 1 diabetes models, hyperglycaemia in Db/Db mice was accompanied by hyperinsulinaemia (Extended Data Fig. 3k), probably ruling out direct insulin impacts on IAV susceptibility. To further extend our findings beyond the context of influenza, we infected hyperglycaemic Akita mice with the respiratory pathogen mouse pneumonia virus (PVM) and noted increased mortality (Fig. 1k), elevated viral titres (Fig. 1k) and increased lung Ifnb1 expression (Extended Data Fig. 3l), coupled with lower numbers of pulmonary T and B cells (Extended Data Fig. 3m–p). Similarly, PVM infection of STZ-treated mice led to reduced viral clearance (Extended Data Fig. 3q) and impaired adaptive immune responses, which were reversed using insulin supplementation (Extended Data Fig. 3r–u). Overall, our findings suggest that pulmonary viral infection in mice featuring types 1 and 2 diabetes-induced hyperglycaemia leads to a marked impairment of lung antiviral adaptive immunity, potentially driving delayed viral clearance and elevated mortality, which is reversible following insulin-mediated glucose lowering.

Hyperglycaemia alters lung DC composition

To investigate the mechanisms driving pulmonary adaptive immune dysfunction and susceptibility to infection during hyperglycaemia, we next performed single-cell RNA sequencing (scRNA-seq) of lung cells from hyperglycaemic Akita mice and normoglycaemic WT littermate controls at both steady-state and two time points during acute IAV infection (1 and 10 d.p.i.). Collectively, we sequenced 154,545 single cells from 24 samples and identified 52 cell types/states across all conditions (Fig. 2a and Supplementary Information 4). During IAV infection we detected influenza transcripts in type 2 alveolar epithelial cells, ciliated epithelial cells and macrophages at both 1 and 10 d.p.i. but not in DC or lymphocytes (Supplementary Information 5a,b). As expected, we noted significant shifts in multiple pulmonary cell subset abundances and transcriptomic profiles during lung infection. Specifically, new subsets of macrophages, DC, natural killer (NK) cells, T cells, B cells, neutrophils and fibroblasts, among others, appeared during infection compared with the non-infected state (Fig. 2b and Supplementary Information 6). Acute IAV infection induced major global transcriptional response shifts in monocytes, alveolar and interstitial macrophages, DC, CD4+ T cells, CD8+ T cells, T Helper 17 (Th17) cells, Treg cells, NK cells, basophils, type  2 alveolar epithelial cells, fibroblasts and endothelial cell subsets (Supplementary Information 7), and higher expression of chemokines including Ccl2, Ccl7 and Ccl8 (Supplementary Information 7b).

Fig. 2: Diabetes alters lung DC in homeostasis and during respiratory viral infection.

a,b, scRNA-seq of lungs during steady-state, 1 d.p.i. and 10 d.p.i. with 50 pfu PR8 (n = 4 per group). a, Uniform manifold approximation and projection (UMAP) of all cells. b, Lung cDC1, two-sided Wilcoxon test. Boxplots show 25th–75th percentiles, the 50th percentile denoted by a thicker line; whiskers show 1.5× interquartile range, or maximum or minimum if smaller than 1.5× interquartile range. Numeral (1) above the horizontal lines denotes the P value. A, Akita. c, Lung cDC1, WT and Akita mice infected with 50 pfu PR8 and treated with Ins/PBS: WT+PBS (n = 6), WT+Ins (n = 7), Akita+PBS (n = 6), Akita+Ins (n = 9), one-way analysis of variance (ANOVA) and Holm–Sidak correction. d, Lung cDC1, WT and Akita mice (n = 7 per group) infected with 50 pfu PVM, two-sided unpaired t-test. e, scRNA-seq of lung DC from PBS- (n = 4) and STZ-administered (n = 4) mice. Enriched Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways in PBS- versus STZ-administered cDC1, P values corrected for multiple hypothesis testing with the g:SCS algorithm. All P values are indicated in Supplementary Table 1. ER, endoplasmic reticulum; EBV, Epstein–Barr virus.

Differential expression profiling between normoglycaemic activated cell types and their corresponding steady-state cells identified hundreds of differentially expressed genes in each comparison (Supplementary Information 7a). Changes in cellular subsets and expression of secreted soluble factors, such as cytokines, were often similar across different cell types (Supplementary Information 7 and 8). Selection of genes featuring differential expression in at least five of ten comparisons showed many Gene Ontology terms related to interferon and immune response in shared upregulated genes, and also terms related to respiration and translation in shared downregulated genes (Supplementary Information 9). Respiration was prominently identified because mitochondrially encoded electron transport chain genes were lower in activated cells (Supplementary Information 10a). This was associated with a generally increased number of transcripts in all cell-activated types, suggesting that it is a relative rather than an absolute decrease in expression (Supplementary Information 10b). Moreover, promoters of commonly upregulated genes exhibited markedly over-represented interferon response factors binding sites, suggesting that interferons constitute key regulators of activation in this setting (Supplementary Information 10c).

We next explored cellular and genomic alterations driven by hyperglycaemia. During steady-state, no major differences in cell numbers were observed between normoglycaemic and hyperglycaemic mice (Fig. 2b and Supplementary Information 6). At 10 d.p.i., hyperglycaemic mice featured significantly fewer lung CD4+ and CD8+ T cells, including activated and cycling populations (Supplementary Information 6). In addition, hyperglycaemic mice harboured reduced numbers of lung follicular B cells (Supplementary Information 6), activated alveolar and interstitial macrophages and NK cells, compared with infected WT littermate controls (Supplementary Information 6). No major differences were noted in most non-haematopoietic cell types, except for a reduction in cycling fibroblasts (Supplementary Information 6). Importantly, infected hyperglycaemic animals evidenced significant reductions in the number of type 1 lung conventional DC (cDC1), including cDC1 cycling cells, CCR7+ migratory DC and activated cDC2, as well as plasmacytoid DC (pDC), compared with infected normoglycaemic littermate controls (Fig. 2b and Extended Data Fig. 4a–h). We did not observe a difference in CD64+ DC, which probably represent a mixture of monocyte-derived DC and recently described inflammatory cDC2 (ref. [10]) (Supplementary Information 6). Differential expression with DESeq2 on pseudobulk counts of infected hyperglycaemic versus normoglycaemic mice showed multiple differentially expressed genes at 10 d.p.i. (Extended Data Fig. 4i,j). Genes higher in hyperglycaemic Akita were mostly related to chemotaxis (Ccl3, Ccl4, Cxcr4) and granzymes (Gzma, Gzmb), whereas those higher in normoglycaemic mice were regulated by interferons (Stat1, Ifit2, Irf7, Gbp) and involved in antigen presentation (Psmb9, B2m, H2-Ab1, H2-K1).

Protein-level validation by flow cytometry-based analysis (Supplementary Information 11) of IAV-infected Akita mice at 10 d.p.i. corroborated that lung cDC1 (Fig. 2c) and, to a lesser extent, classical cDC2 and CD64+ DC, were reduced in lungs of infected hyperglycaemic mice compared with littermate controls (Extended Data Fig. 5a,b). Furthermore, we observed reduced expression of IL-12 and Ki-67+ in cDC1 from hyperglycaemic mice during IAV infection (Extended Data Fig. 5c,d). Interestingly, also at steady-state, lung DC subset numbers were significantly lower in hyperglycaemic Akita mice (Extended Data Fig. 5e and Supplementary Information 11) and were associated with a lower expression of Ki-67 (Extended Data Fig. 5f), suggesting that already in naïve mice lung DC may be developmentally impaired. Indeed, bone marrow analysis of naïve Akita mice showed a moderate reduction in pre-DC and common DC progenitors (Extended Data Fig. 5g), most notably pre-DC1 (Extended Data Fig. 5h). We did not observe an impact of hyperglycaemia on circulating lymphocytes (Extended Data Fig. 5i). Similar to type 1 diabetes models, Db/Db mice also exhibited a reduction in conventional lung DC but not CD64+ DC (Extended Data Fig. 5j,k) and lower frequency of Ki-67+ cells (Extended Data Fig. 5l), suggesting that it is not the absence of insulin, but rather hyperglycaemia, that probably drives lung DC aberrations. In corroboration of these influenza-related findings, lung DC in hyperglycaemic mice infected with PVM evidenced a depletion in lung cDC1, cDC2 and pDC (Fig. 2d and Extended Data Fig. 5m,n), coupled with fewer Ki-67+ DC and lower IL-12 expression in lung cDC1 (Extended Data Fig. 5o,p), compared with PVM-infected, normoglycaemic mice. Importantly, impairment in lung DC in virally infected Akita mice was rescued by insulin-driven lowering of hyperglycaemia (Fig. 2c and Extended Data Fig. 5a,b). Interestingly, even in non-infected, hyperglycaemic Akita mice, glucose normalization led to replenished numbers of pulmonary cDC1 and cDC2, and Ki-67+ cDC1 (Extended Data Fig. 5q–s), but not of steady-state expression of IL-12 (Extended Data Fig. 5t). In agreement, insulin-mediated correction of hyperglycaemia in IAV-infected, STZ-treated mice reversed the altered total lung DC composition (Extended Data Fig. 6a–c), cDC1 Ki-67+ cells (Extended Data Fig. 6d) and IL-12 expression (Extended Data Fig. 6e). Similarly, insulin supplementation reversed lung DC alterations in STZ-treated, PVM-infected mice (Extended Data Fig. 6f–h).

In corroberation of these findings, scRNA-seq of enriched lung DC from naïve STZ-treated mice and controls identified the same six DC populations that we observed in our previous non-enriched dataset, including pDC, cycling and non-cycling cDC1, cDC2, CD64+ DC DC and migratory CCR7+ DC (Extended Data Fig. 6i,j). Alike our previous findings, cDC1 were the most significantly impaired pulmonary immune cell population in hyperglycaemic mice, even at steady-state (Extended Data Fig. 6k–p). Differential expression analysis identified 320 differentially expressed genes in CD64+ DC, 115 in cDC1, one in cDC2 and 14 in pDC (Extended Data Fig. 7a–e). Across populations, DC from hyperglycaemic mice exhibited lower levels of genes related to antigen presentation (Fig. 2e and Extended Data Fig. 7f) and to terpenoid and steroid biosynthesis in CD64+ DC (Extended Data Fig. 7g). Overall, hyperglycaemia induced by different models was associated with major alterations of lung DC in steady-state and during viral infection, manifesting as compositional and gene expression aberrations in multiple lung DC subsets, most notably cDC1. DC impairment was rescued by insulin-based normalization of hyperglycaemia.

Hyperglycaemia alters lung DC function

We next studied the functional consequences of hyperglycaemia on lung DC function. During an encounter with a pathogen, lung DC engulf and transport antigens to the lung-draining lymph nodes (dLN), where they prime naïve T cells. Indeed, simultaneous injection of the fluorescent antigen ovalbumin (OVA)-AF647 into IAV-infected Akita hyperglycaemic mice and normoglycaemic littermate controls showed that the frequency and number of DC carrying OVA in lung dLN were reduced in hyperglycaemic mice (Fig. 3a and Extended Data Fig. 8a–c). By contrast, OVA uptake was enhanced in lung DC that had not migrated to lung dLN (Extended Data Fig. 8d). Furthermore, expression levels of the key lung DC costimulatory molecules CD40, CD80 and CD86, which are required for effective antigen presentation to T cells, were lower in total dLN DC in infected hyperglycaemic animals (Fig. 3b–d and Extended Data Fig. 8e–g) and, to a lesser extent, also in non-migratory lung DC (Extended Data Fig. 8h–j) compared with infected normoglycaemic controls. To further demonstrate that these key lung DC functions are directly impacted by hyperglycaemia, we isolated lung DC from naïve WT animals and incubated them with either high (50 mM) or normal (10 mM) levels of glucose (Supplementary Information 12). Although high glucose had no impact on cell viability (Extended Data Fig. 8k), it directly reduced expression of costimulatory molecules in cDC1 and cDC2 compared with normal glucose controls (Fig. 3e–g and Extended Data Fig. 8l–n). Exposure to intermediate concentrations of excessive glucose (25 mM) yielded similar effects (Extended Data Fig. 8o–t).

Fig. 3: High glucose impairs lung DC function.

a–d, WT (n = 9) and Akita (n = 6) mice infected with 50 pfu PR8 and intratracheally administered 100 μg of OVA-AF647, two-sided unpaired t-test. a, Lung dLN OVA+ cDC1. b, CD40 mean fluorescence intensity (MFI). c, CD80 MFI. d, CD86 MFI. e–g, WT lung cDC1 incubated with high (50 mM, n = 4) or normal (10 mM, n = 4) glucose. e, CD40 MFI, two-sided unpaired t-test. f, CD80 MFI, two-sided Mann–Whitney U-test. g, CD86 MFI, two-sided unpaired U-test. h, WT lung cDC1 incubated with high (50 mM, n = 8) or normal (10 mM, n = 9) glucose for 20 h, then co-incubated for 4 days with OT-I-CD8+ T cells in normal (10 mM) glucose. CD8+ T cells, two-sided unpaired t-test. i, WT lung cDC1 incubated with high (50 mM, n = 12) or normal (10mM, n = 12) glucose for 20 h, then co-incubated for 4 days with OT-II-CD4+ T cells in normal (10 mM) glucose. CD4+ T cells, two-sided unpaired t-test. All P values are indicated in Supplementary Table 1. All data mean+s.e.m.

To generalize these hyperglycaemia-induced functional lung DC impairments in antigen transport and costimulation beyond the pulmonary viral infection context we used the house dust mite (HDM) model, a common model of allergic inflammation mimicking asthma, which critically involves cDC activation and adaptive immune instruction[11]. Similar to IAV infection, administration of HDM extract together with OVA-AF647 to hyperglycaemic mice led to unaffected antigen uptake and reduced antigen transport, coupled with lowered expression of several costimulatory molecules in both migratory and lung-resident DC as compared with non-hyperglycaemic littermate controls (Extended Data Fig. 8u-ab). Furthermore, HDM challenge of hyperglycaemic mice was associated with protection from key asthma hallmarks, including eosinophilia and excessive mucus production, in both the Akita and STZ hyperglycaemia models compared with non-hyperglycaemic controls (Extended Data Fig. 9a–e).

To evaluate the functional consequences of high glucose on DC-mediated T cell priming, WT lung DC from normoglycaemic animals were incubated in vitro with high (50 mM) or normal (10 mM) glucose and OVA for 20 h, followed by co-incubation with OT-I CD8+ T cells in normal (10 mM) glucose medium (Supplementary Information 12b). Following 4 days of coculture, DC previously incubated in high glucose exhibited an impaired capacity to induce T cell expansion compared with those in normal glucose (Fig. 3h and Extended Data Fig. 9f), suggesting that high glucose had directly suppressed their capacity to activate T cells. Similarly, incubation of DC in high glucose impaired their capacity to activate OT-II CD4+ T cells (Fig. 3i and Extended Data Fig. 9g), whereas high glucose had no detrimental effect on direct anti-CD3- and anti-CD28-mediated T cell activation (Extended Data Fig. 9h,i). To corroborate these findings, we generated Zbtb46-DTR Bone Marrow (BM) chimeric mice in which a specific depletion of DC is achieved following diphtheria toxin treatment. In this model, DC-depleted, IAV-infected mice exhibited impaired antiviral immunity (Extended Data Fig. 9j–r) similar in magnitude and not further exacerbated by the co-induction of STZ-mediated hyperglycaemia.

To evaluate whether hyperglycaemia directly impairs the ability of lung DC to induce adaptive immunity in vivo, we intratracheally immunized hyperglycaemic Akita mice and normoglycaemic littermate controls with the innocuous antigen HDM, followed by isolation of lung DC 1 day after treatment and their transfer to naïve WT normoglycaemic recipients. Subsequent HDM challenge in recipients showed that lung DC transferred from hyperglycaemic animals poorly induced lung inflammation, leading to lower infiltration of granulocytes, T cells and monocytes, as compared with lung DC transferred from normoglycaemic donors (Extended Data Fig. 9s–w and Supplementary Information 13). Consequently, recipients of lung DC from hyperglycaemic animals developed marked alterations in their pulmonary adaptive immune response, manifesting as a lower frequency of Ki-67+CD8+ T cells, CD4+ T cells, and Th2 cytokine-producing cells including GATA3+CD4+ T cells, as compared with recipients of DC from normoglycaemic donors (Extended Data Figs. 9x and 10a–d). To validate these findings in the context of a respiratory viral infection, while avoiding concomitant transfer of live virions with isolated donor DC, we administered WT and diabetic Akita mice with ultraviolet-inactivated PR8 influenza virus, followed, 24 h later, by transfer of lung DC to naïve WT normoglycaemic recipients. Indeed, subsequent challenge of recipient mice with a high viral dose showed impaired capacity of DC from diabetic animals to induce a CD4+ and CD8+ T cell response (Extended Data Fig. 10e,f), which was associated with an elevated viral titre compared with animals receiving DC from normoglycaemic donors (Extended Data Fig. 10g). Collectively, these results suggest that hyperglycaemia drives functional defects in lung DC, resulting in major downstream effects on adaptive lung immune activation and function in both infectious and non-infectious inflammatory settings.

Hyperglycaemia alters lung DC metabolism

We next sought to elucidate the underlying mechanisms orchestrating hyperglycaemia-induced lung DC functional impairment. Given the emerging links between glucose metabolism and immune function in immune cells such as macrophages[9] or T cells[12], we hypothesized that high glucose levels could lead to alterations in lung DC glycolysis and downstream metabolic circuits which, in turn, could affect their immune function. Several important hints suggested such a link. First, murine WT lung DC expressed the enzyme machinery necessary for glycolysis (Supplementary Information 14a). Second, our single-cell transcriptomic analysis demonstrated marked changes in expression of genes involved in glucose metabolism in lung DC of hyperglycaemic animals, including increased expression of phosphofructokinase (Pfkp), fructokinase (Khk), pyruvate dehydrogenase (Pdk1) and pyruvate transporter (Mpc2), and decreased expression of hexokinase (Hk2), in different DC subpopulations (Supplementary Information 14b–f). Third, inhibition of glycolysis in normoglycaemic or hyperglycaemic DC by in vitro incubation with the glycolysis inhibitor 2-deoxyglucose (2-DG) at non-toxic concentrations (Extended Data Fig. 10h) induced impaired expression of costimulatory molecules (Extended Data Fig. 10i–n), IL-12 (Extended Data Fig. 10o), T cell induction and cytokine production (Extended Data Fig. 10p–u). In vivo, daily 2-DG treatment of IAV-infected mice led to transient hyperglycaemia after each injection (Extended Data Fig. 10v), impaired survival (Extended Data Fig. 11a), delayed viral clearance (Extended Data Fig. 11b), impaired adaptive immunity (Extended Data Fig. 11c–h) and reduced lung cDC1 (Extended Data Fig. 11i–k). 2-DG-treated naïve mice demonstrated a similar phenomenon in DC (Extended Data Fig. 11l,m) but not in T cells (Extended Data Fig. 11n).

For functional evaluation of the potential impacts of hyperglycaemia on glycolysis, we measured the extracellular acidification rate (ECAR; Seahorse) of lung DC obtained from hyperglycaemic Akita mice and normoglycaemic WT controls. Changes in extracellular pH were used as a proxy for glycolysis-mediated lactate production. Importantly, a reduced ECAR was noted in lung DC from hyperglycaemic mice, suggesting potential impairment of lactate production compared with controls despite higher availability of a glucose substrate during hyperglycaemia (Extended Data Fig. 11o,p). No significant impact was observed on oxygen consumption rate, suggesting that mitochondrial respiration remained unaffected (Extended Data Fig. 11q). In addition, no diabetes-induced changes were noted in expression of enzymes required for lipid beta-oxidation (Extended Data Fig. 11r,s). To corroborate the unexpected reduction in ECAR, lung DC isolated from normoglycaemic and hyperglycaemic mice were incubated ex vivo with 13C-glucose. Tracing of isotope-labelled glucose utilization to 13C-labelled lactate in the supernatant showed lower levels of lactate produced by cells from hyperglycaemic animals compared with controls (Fig. 4a), which was associated with lower intracellular levels of the glycolytic metabolite pyruvate following 6 h of incubation (Extended Data Fig. 11t). Neither lung DC, nor other tested cell types from hyperglycaemic animals exhibited a deficiency in uptake of the glucose analogue 2-NBDG, suggesting that, at the in vivo setting, there is no difference in glucose uptake by immune cells under hyperglycaemic conditions (Extended Data Fig. 12a–c and Supplementary Information 15).

Fig. 4: Hyperglycaemia dysregulates lung DC metabolism and acetylation.

a, WT (n = 4) and Akita (n = 4) lung DC incubated with 11 mM 13C-glucose. Supernatant 13C-lactate, area under the curve, two-sided unpaired t-test. b, WT (n = 15) and Akita (n = 10) lung DC acetyl-CoA, two-sided unpaired t-test. c, WT lung cDC1 incubated with high (50 mM, n = 12)/normal (10 mM, n = 9) glucose or with BMS303141 in high (50 mM, n = 16)/normal (10 mM, n = 13) glucose, then co-cultured for 4 days with OT-I-CD8+ T cells in normal (10 mM) glucose (in the absence of inhibitor). CD8+ T cells, two-way ANOVA with Holm–Sidak correction. d, WT lung cDC1 incubated for 20 h with 10 mM dichloroacetate (DCA) (n = 12) or medium (n = 12), then co-cultured for 4 days with OT-I-CD8+ T cells in normal (10 mM) glucose (in the absence of inhibitor). CD8+ T cells, two-sided Mann–Whitney U-test. e, OT-I-CD8+ T cells incubated for 4 days with lung WT cDC1 (n = 15) or Pdk2–4−/− cDC1 (n = 11). CD8+ T cells, two-sided unpaired t-test. f, H3K27ac immunoblot of lung DC from WT (n = 6) and Akita (n = 5) mice, two-sided unpaired t-test. g, Fluorescent activated cell sorting of lung WT (n = 6) and Akita (n = 6) H3K27ac+ DC, two-sided unpaired t-test. h, WT lung cDC1 incubated for 20 h with high (50 mM, n = 6)/normal (10 mM, n = 8) glucose, or with 10 mM ANA in high (50 mM, n = 8)/normal (10 mM, n = 8) glucose, then co-cultured for 4 days with OT-I-CD8+ T cells and normal (10 mM) glucose (in the absence of inhibitor). CD8+ T cells, two-way ANOVA with Holm–Sidak correction. i,j, WT and Akita mice intraperitoneally administered 5 mg kg−1 ANA or vehicle (dimethyl sulfoxide, DM) for 5 days, followed by intratracheal administration of 50 μg of OVA + 50 μg poly I:C. 24 h later, lung DC were sorted and transferred to naïve WT mice: WT+DM DC (n = 28), Akita+DM DC (n = 23), Akita+ANA DC (n = 12 for i, n = 11 for j). 10 days later, recipient mice were administered 500 pfu PR8-OVA (SIINFEKL), analyzed at day 7, Kruskal–Wallis test with Dunn’s correction. i, Lung OVA-PR8 viral RNA. j, Lung CD8+ T cells. All P values are indicated in Supplementary Table 1. All data mean+s.e.m.

Importantly, elevated acetyl-CoA levels were measured in lung DC from hyperglycaemic compared with control mice (Fig. 4b), suggesting a potential rerouting of glucose metabolism away from lactate production towards acetyl-CoA during hyperglycaemia. To further investigate whether acetyl-CoA accumulation in hyperglycaemia-exposed lung DC might be linked directly to their capacity to activate T cells, we inhibited acetyl-CoA production using treatment with BMS303141, an inhibitor of ATP citrate lyase, the key enzyme converting citrate to acetyl-CoA in the tricarboxylic acid cycle[13]. Indeed, ATP citrate lyase inhibition during hyperglycaemia partially restored the capacity of lung cDC1 and cDC2 to induce T cell expansion and expression of the proliferation marker Ki-67 (Fig. 4c and Extended Data Fig. 12d–f). To further corroborate a potential detrimental role of acetyl-CoA accumulation in lung DC function we inhibited pyruvate dehydrogenase kinases (PDKs), critical negative regulators of the generation of acetyl-CoA, thereby leading to accumulation of acetyl-CoA[14]. Indeed, preincubation of lung DC with a non-toxic concentration (10 mM) of the global PDK-inhibitor dichloroacetate, followed by the addition of OT-I CD8+ T cells after removal of the inhibitor, reduced the capacity of cDC1 and cDC2 to induce T cell expansion (Fig. 4d and Extended Data Fig. 12g) and IFNγ production (Extended Data Fig. 12h). To corroborate these findings, we sorted lung DC from Pdk2/3/4-deficient mice and repeated the above cocultures with OT-I CD8+ T cells. Indeed, Pdk2/3/4-deficient DC induced altered CD8+ T cell expansion (Fig. 4e and Extended Data Fig. 12i), IFNγ production and expression of Ki-67 (Extended Data Fig. 12j,k) compared with WT controls. Collectively, our results suggest that hyperglycaemia probably alters the metabolic state of lung DC through excessive generation of acetyl-CoA, thereby leading to impairment of lung DC function. Inhibition of key steps in acetyl-CoA production reversed these aberrations.

Altered chromatin landscape in lung DC

Increased acetyl-CoA levels in lung DC of diabetic animals may lead to epigenetic modifications, by potentially modulating histone acetylation which, in turn, plays a major role in chromatin remodelling and global gene expression[13]. Indeed, lung DC in diabetic and normoglycaemic animals broadly expressed the enzyme machinery that regulates epigenetic modifications (Supplementary Information 16a). To explore this possibility, we utilized the CUT&Tag epigenetic profiling of lysine 27 histone 3 modifications in DC from naïve WT and hyperglycaemic Akita mice. We selected trimethylation and acetylation at the same amino acid, reasoning that these modifications cannot co-occur, thereby enabling quantitative determination of global hyperglycaemia-induced genomic effects on lung DC. We noted a global shift from methylation to acetylation in DC obtained from hyperglycaemic animals as compared with those obtained from normoglycaemic controls (Extended Data Fig. 12l,m and Supplementary Information 16b). Immunoblotting supported elevated H3K27 acetylation (H3K27ac) in DC from hyperglycaemic mice (Fig. 4f and Supplementary Information 1). Moreover, we identified six differentially abundant peaks higher in acetylation and 60 in the methylation data, between hyperglycaemic and normoglycaemic conditions, including peaks approximating key immune genes such as Cx3cl1, Fcrl5, Acvr1, Rora and Il17a, genes coding for metabolic enzymes such as Pfkfb4, Eci2, Cox6c2 and Dgat2 and also Hdac4 (Supplementary Information 16b). In agreement, flow cytometric analysis showed elevated H3K27ac in DC subsets from diabetic animals (Fig. 4g and Extended Data Fig. 12n). These findings indicated that hyperglycaemia probably induces global impacts on the chromatin state of lung DC.

Epigenetic DC modulation in diabetes

Finally, we aimed to elucidate whether manipulation of the identified metabolic–epigenetic–immune DC axis could attenuate hyperglycaemia-induced lung DC dysfunction, immune impairment and exacerbated pulmonary viral disease. We began by isolation of primary lung DC from naïve WT normoglycaemic mice and incubating them with normal (10 mM) or high (50 mM) glucose, while adding anacardic acid (ANA), an inhibitor of histone-acetyl transferase that blocks excessive acetylation. Abrogation of acetylation in DC exposed to high glucose levels improved their capacity to prime CD8+ T cells, increased the frequency of proliferating Ki-67+CD8+ T cells and promoted their capacity to produce IFNγ (Fig. 4h and Extended Data Fig. 12o–s).

To determine the in vivo impact of hyperacetylation inhibition on altered lung DC function during respiratory infection, we treated WT and Akita mice with ANA before and during influenza virus infection. Analysis at 10 d.p.i. showed that histone-acetyl transferase inhibition in hyperglycaemic mice led to a rescue of lung cDC1, cDC2 and CD64+ DC (Extended Data Fig. 12t–v), coupled with a mild increase in Ki-67+ cDC1 (Extended Data Fig. 12w). Similarly, ANA treatment led to lung DC expansion in the STZ-induced diabetes model (Extended Data Fig. 12x–z). To assess the in vivo consequences of the reversal of lung DC hyperacetylation on CD8+ T cell immunity, we treated WT and diabetic Akita mice with ANA and then immunized them with OVA and polyinosinic acid:polycytidylic acid (poly I:C), mimicking a viral infection. Lung DC were then sorted 24 h post injection and transferred to naïve WT recipients, which were challenged, 10 days later, with the PR8-OVA(SIINFEKL) influenza virus. Analysis 7 days following infection showed improved viral clearance (Fig. 4i), coupled with enhanced induction of CD8+ T cell immunity (Fig. 4j), in mice that had received lung DC from ANA-treated hyperglycaemic mice compared with recipients of lung DC from vehicle-treated hyperglycaemic controls. Taken together, our results suggest that reversal of hyperglycaemia-induced lung DC hyperacetylation and associated epigenetic alterations may enable rescue of their ability to prime T cells, thereby constituting a potential means of ameliorating diabetes-associated immune defects.

Discussion

Our results provide several conceptual advances. We describe the cellular and transcriptomic changes occurring during respiratory viral infection at the single-cell level (performed in 154,545 cells) and establish glucose as a critical metabolic regulator of lung DC function in steady-state and during viral infection. This result is in line with emerging evidence suggesting that metabolism impacts the function of other immune cells, such as T cells and macrophages15,16. We hypothesize that glucose regulation may constitute a host resistance mechanism against pathogens[17], in which the host maintains a critical balance between the need to limit pathogen access to essential energy sources, and also fuelling protective immune responses[17]. Altered regulation of glucose control in diabetes disrupts this delicate host–microbe balance, thereby leading to enhanced susceptibility to viral infection. Similar principles may contribute to increased diabetes-related risks in viral, bacterial and fungal infections occurring in the respiratory tract and other mucosal surfaces. These merit future research. Likewise, the role of glucose in modulation of DC bone marrow development and tissue maintenance[18] warrants future investigation.

Metabolic DC reprogramming, mainly studied in cancer and embryonic development, constitutes a critical mechanism of DC immune regulation[19] and is mediated by a variety of signals such as TLR[20], type I IFN[21] and downstream acetyl-CoA-associated histone hyperacetylation[13]. Although previous studies highlighted hyperacetylation as a driver of gene activation in specific genomic loci[22], hyperglycaemia-induced hyperacetylation may lead to the repression of immune response-related genes, possibly through chromatin destabilization and dominant aberrations in gene expression[23]. Altered H3K27 trimethylation may have lesser impacts on DC in other tissues[24]. Of note, in some contexts, H3K27 hyperacetylation may be coupled with suppressed methylation[25], whereas in others the two processes (regulated by different mechanisms) could occur independently of each other, meriting further studies in the diabetes context. Other glucose-related processes, such as protein glycosylation or internal cell glycogen storage[26], may also be affected by hyperglycaemia, thereby constituting interesting topics for future research.

From a translational point of view, the reconstituted antiviral immunity induced by insulin-driven correction of hyperglycaemia highlights the importance of a meticulous and proactive tight glucose control strategy in diabetics, including during acute infection. Beyond glucose control, checkpoint inhibition of aberrant DC acetylation may prevent or treat hyperglycaemia-induced immune dysfunction and its clinical ramifications. Indeed, inhibition of histone acetylation-regulating enzymes is actively explored as a treatment for cancer and other diseases involving aberrant acetylation[27]. Local administration of acetylation-targeting interventions (through inhalation) may provide an opportunity for effective rescue of lung DC immune function during pulmonary infection in diabetes, while minimizing systemic off-target effects. Putative adverse effects of such immune-reconstituting treatment, including cytokine release and chronic inflammation, merit further exploration. Likewise, validation of our findings in human diabetics inflicted with an acute pulmonary infection, such as influenza or SARS-CoV2 (ref. [28]), merits future studies.

Methods

Mice

WT C57BL/6 male mice (Harlan), Akita mice[29] (Jackson strain no. 003548) and their WT littermate controls, Db/Db (Jackson strain no. 000697), OT-I (C57BL/6-Tg (TcraTcrb)1100Mjb/J), OT-II (B6.Cg-Tg(TcraTcrb)425Cbn/J), Zbtb46-DTR and Pdk2/3/4−/− mice[30] were bred and maintained at the Weizmann Institute of Science animal facility under standard day/night cycles at room temperature and normal humidity and had access to ad libitum food and water. WT littermates served as controls. Animals were 8–14 weeks of age and were randomly assigned to groups. All experiments were performed in accordance with institutional guidelines and were approved by the Weizmann Institute of Science Institutional Animal Care and Usage Committee (approval nos. 05400622-2, 02800321-1, 04000520-2 and 14760619-3).

STZ-induced diabetes

Mice received 100 mg kg−1 STZ (Sigma) in PBS or PBS as vehicle control via an intraperitoneal injection on days 0 and 1 and were then allowed to recover for 2 weeks before further experimental steps[31]. Hyperglycaemia in STZ-injected mice was confirmed before every experiment.

Viral infection

We used influenza virus strain PR8 (A/Puerto Rico/34, H1N1) or OVA (SIINFEKL)-PR8 (ref. [32]). For PVM, we used VR-1819 from the American Type Culture Collection. Mice were anaesthetized using isoflurane and then intratracheally infected with 50 µl of virus in PBS at the indicated doses. Animals were monitored daily and euthanized if they fulfilled severity criteria set out by the institutional guidelines.

Detection of virus-specific antibodies

Blood was collected retro-orbitally using heparin-coated glass capillaries, transferred to heparin-coated tubes and kept on ice until centrifugation for 15 min at 10,000g and 4 °C. Serum was transferred to fresh tubes and stored at −80 °C until use. BAL was collected by insertion of a catheter into the trachea and flushing the lungs with 1 ml of PBS. Samples were stored on ice until centrifugation for 5 min at 500g to remove the cellular fraction. BAL fluid was stored at −80 °C until further use. For detection of antiviral antibodies, serum and BAL fluid were measured for virus-specific IgM and IgG2b antibody levels; 96-well plates (Maxisorp, Nunc) were coated with ultraviolet-inactivated influenza virus (PR8) in PBS overnight at 4 °C. Plates were washed and incubated with PBS-1% bovine serum albumin (BSA) for 2 h at room temperature for blocking. BAL fluids from individual mice were serially diluted in PBS with 0.1% BSA, starting with a 1:3 dilution for BAL fluids and 1:50 dilution for sera, followed by incubation at room temperature for 2 h. Plates were washed five times and incubated with alkaline phosphate-labelled goat anti-mouse antibodies to IgM or IgG2b (Southern Biotech Technologies, Inc.) at 1:1,000 dilution in PBS with 0.1% BSA at room temperature for 2 h. Thereafter, plates were washed five times and substrate p-nitrophenyl phosphate (Sigma-Aldrich) was added. Optical densities were measured on an enzyme-linked immunosorbent assay reader (Bucher Biotec) at 405 nm. For analysis, antibody titre data were analysed by fitting function y = a × x/(b + x) + c to each sample using nonlinear least-squares regression model, where x is 1/antibody dilution, a is the saturation signal in ELISA, b is the concentration of the antibody required to achieve 50% of saturation signal in ELISA and c is the background signal. Half-maximal effective concentrations (inflection points), which are indicative of the concentration of the antibody required to achieve 50% of saturation signal in ELISA, were then used to perform statistical testing.

Flow cytometry

Mice were euthanized by intraperitoneal injection of 200 mg ml−1 sodium pentobarbital. Lungs were perfused with cold PBS and put on ice after removal. LdNs were collected and then digested with 2 mg ml−1 type IV collagenase (Worthington) and 1 mg ml−1 DNase I (Sigma) at 37 °C for 20 min in Iscove’s modified Dulbecco’s medium (IMDM), and subsequently passed through a 70 µm cell strainer using 10 ml of PBS. Lungs were minced and digested with 1 mg ml−1 Hyaluronidase (Sigma), 25 μg ml−1 Collagenase XI (Sigma), 50 μg ml−1 Liberase TM (Roche) and 1 mg ml−1 DNase I (Sigma) in IMDM at 37 °C for 30 min, and subsequently passed through a 70 µm cell strainer using 20 ml of PBS. Cells were then centrifuged for 10 min at 500g and resuspended with 10 ml of BS. Cells were centrifuged for 7 min at 500g before resuspension in 1 ml of PBS. Samples were stained using Zombie viability dye (BioLegend) according to the manufacturer’s instructions. Fc receptors were blocked using 1 μg ml−1 anti-CD16/32 (BioLegend) 1:200. Cells were stained with the following anti-mouse antibodies at the indicated dilutions: CD11c APC-Cy7 (N418) 1:200, CD11b BV605 (M1/70) 1:1,000, Siglec-F PE (E50-2440, BD Biosciences) 1:1,000, CD45 AF700 (30-F11) 1:1,000, CD45 BV711 (30-F11) 1:1,000, CD45.2 (104) BV711 1:200, CD4 PerCP-Cy5.5 (GK1.5) 1:200, CD4 (GK1.5) FITC 1:200, CD8 (53-6.7) BV605 1:200, CD8 (53-6.7) PE 1:1,000, IL-4 PE (11B11) 1:200, IL-5 BV421 (TRFK5) 1:200, IFNγ PE-CF594 (XMG1.2) 1:500, Ly-6G PerCP-Cy5.5 (1A8-Ly-6g) 1:1,000, MHC class II BV421 (M5/114.15.2) 1:1,000, CD64 APC (X54-5/7.1) 1:1,000, CD19 BV605 (6D5) 1:200, CD19 (6D5) APC (1:200), CD90.2 BV605 (30-H12) 1:1,000, TCR-β APC-Cy7 (H57-597) 1:200, TCRγδ PerCP-Cy5.5 (GL3) 1:200, NK1.1 APC (PK136, eBioscience) 1:200, FcεRIα PE_CF594 (MAR-1, Thermo Fisher Scientific) 1:1,000, XCR1 BV510 (ZET) 1:500, GATA3 BV421 (16E10A23) 1:100, RORγt PE (AFKJS-9, Thermo Fisher Scientific) 1:1,000, IL-17A PerCP-Cy5.5 (TC1118H10.1) 1:200, IL-13 AF488 (eBioscience, eBio13A) 1:200, T-bet PE_Cy7 (4B10) 1:800, FoxP3 PE (MF-14) 1:200, CD40 (3/23) PerCP 1:100, CD80 PE-Cy7 (16-10A1) 1:200, CD86 PE-CF594 1:1,000 (GL-1), F4/80 (BM8) PE-Cy7 1:200, Ly-6C PE-Cy7 (HK1.4) 1:1,000, all purchased from BioLegend unless otherwise stated. BV421-conjugated peptide-MHC class I tetramers (H-2Db/NP34), with the NP34 peptide (NP366-374, ASNENMETM) from the nucleoprotein of influenza virus A/PR/8/34, were obtained from the NIH tetramer core facility and used at 1:400. For sorting experiments, dead cells were excluded using the live/dead marker DAPI (BioLegend, 20 ng ml−1). For intracellular staining of transcription factors or histone modifications, cells were fixed and stained using the intracellular fixation and permeabilization kit (eBioscience) according to the manufacturer’s instructions. For staining of H3K27ac, cells were stained with primary rabbit anti-H3K27ac antibody (abcam, 1:400) followed by secondary goat anti-rabbit AF647 (Invitrogen, 1:1,000). For cytokine analysis, T cells were restimulated with a cell activation cocktail (BioLegend) containing Brefeldin A for 4 h. For cytokine analysis of myeloid cells, cells were incubated for 4 h with Brefeldin A only. Flow cytometry analysis was performed on a LSR Fortessa (BD) or ARIA III (BD) using DIVA software (BD) and analysed with FlowJo (Tree Star). Cells were identified in the following way: CD4+ T cells as CD45+TCRb+CD4+, CD8+ T cells as CD45+TCRb+CD8+, B cells as CD45+CD19+TCRb−, eosinophils as CD45+CD11c−CD11b+Siglec-F+SSC-Ahigh, neutrophils as CD45+CD11c−CD11b+Ly-6G+, lung cDC1 as Siglec-F−MHCII+CD11c+XCR1+, lung cDC2 as Siglec-F−MHCII+CD11c+XCR1−CD11b+CD64−, lung CD64+ DC as Siglec-F−MHCII+CD11c+XCR1−CD11b+CD64+, dLN cDC2 as CD45+autofluorescent−CD11c+MHCIIhighXCR1+, lung dLN cDC1 as CD45+autofluorescent−CD11c+MHCIIhighXCR1−CD11b+CD64−, Ly-6Chigh monocytes as Siglec-F−Ly-6G−CD11b+Ly-6Chigh and Ly-6Clow monocytes as Siglec-F−Ly-6G−CD11b+Ly-6Clow.

Ex vivo DC experiments

Lung DC were isolated from WT mice using the tissue preparation protocol described above. Following the establishment of a single-cell suspension, cells were incubated for 1 h on ice with anti-MHCII beads (Miltenyi) and the enriched fraction was collected using LS MACS columns (Miltenyi). Cells were then stained and cDC1 and cDC2 were sorted. DC were then incubated in RPMI 1640 (without glucose) with 20% fetal calf serum, glutamine, HEPES, penicillin/streptomycin and low (10 mM) or high (50 mM) glucose for 20 h before either analysis or replacement with fresh low-glucose medium and the addition of T cells (see below). Where indicated, either 2-DG (5 mM; Sigma), BMS303141 (10 mM; Sigma) or ANA (20 mM; Sigma) was added.

Seahorse

Cells were seeded at 1 × 10^5 cells per well in an Xfe96 analyser (Agilent) and ECAR or oxygen consumption rate was measured over 80 min with the addition of 10 mM glucose, 1 µM oligomycin and 50 mM 2-DG (for ECAR) and 1 µM oligomycin, 1 µM carbonyl cyanide 4-(trifluoromethoxy) phenylhydrazone and 0.5 µM actinomycin and rotenone at the indicated time points.

Ex vivo metabolite analysis

Cells (1–2 × 10^5) were incubated in RPMI medium (Biological Industries, no. 01-101-1 A) with 10% dialysed FBS (Gibco), 4 mM glutamine and 11 mM U-13C6d-glucose (Cambridge Isotope Laboratories, no. 1396) for 6 h. At every time point, 22 µl of medium was briefly centrifuged to remove cell debris and snap-frozen in liquid nitrogen. Medium (20 µl) from each time point was extracted with 400 µl of ice-cold 80:20 methanol:water with 2 µg ml−1 ribitol as an internal standard (extraction solvent), vortexed and immediately centrifuged at 18,000g for 15 min at 4 °C. Supernatants were collected and dried by vacuum (Speed Vac and lyophilizer). Dried samples were incubated with 20 µl of methoxyamine hydrochloride (Alfa Aesar, no. A19188; 20 mg ml−1 in pyridine) at 37 °C for 90 min with shaking, followed by incubation with 35 µl of N, O-bis trimethylsilyl trifluoroacetamide (Sigma, no. 15222) at 37 °C for 30 min. Gas chromatography–mass spectrometry was performed using gas chromatograph no. 7820AN (Agilent Technologies) interfaced with mass spectrometer no. 5975 (Agilent Technologies), with a HP-5ms capillary column 30 m 250 mm 0.25 mm (no. 19091S-433, Agilent Technologies). Helium carrier gas was flowed at a constant rate of 1.0 ml min−1. Gas chromatograph column temperature was programmed from 70 to 150 °C via a ramp of 4 °C min−1, from 250 to 215 °C via a ramp of 9 °C min−1, from 215 to 300 °C via a ramp of 25 °C min−1 and maintained at 300 °C for 5 min. Mass spectrometry was performed by electron impact ionization and operated in full-scan mode from 30 to 500 m/z. Inlet and mass spectrometer transfer line temperatures were 280 °C and ion source temperature was 250 °C. Sample injection (1 µl) was in splitless mode. Raw data signals obtained from gas chromatography–mass spectrometry were analysed using MassHunter software (Agilent Technologies). Isotopologue distribution of lactate was corrected for naturally occurring isotopes using IsoCor software[33].

scRNA-seq

Animals were euthanized by intravenous pentobarbital and lungs perfused with cold PBS. Subsequently, 1 ml of 5 mg ml−1 dispase solution (Sigma) was injected through a tracheal catheter. The solution was incubated for 5 min before the lungs were excised and placed in IMDM (Gibco) on ice. Lungs were then minced and incubated with an enzyme mix containing 1 mg ml−1 DNase I (Sigma), 2 mg ml−1 Collagenase IV (Worthington) and 5 mg ml−1 dispase (Sigma) for 20 min in a shaker at 37 °C. The digested extract was then transferred to a 70 µm cell strainer and smashed through with 20 ml of PBS to obtain a single-cell suspension. Cells were centrifuged at 4 °C for 10 min at 500g. The pellets were resuspended in 1 ml of ACK buffer (Gibco, no. A1049201) and incubated for 1 min at room temperature, followed by the addition of 14 ml of PBS, and cells were centrifuged at 500g for 7 min. Cells were incubated for 1 h on ice in 500 µl of dead cell depletion kit beads (Miltenyi), after which 5.5 ml of kit buffer was added to each sample. Samples were filtered and loaded onto LS columns (Miltenyi) and the flowthrough collected. For experiments using cellular fractions enriched for DC, 50 µl each of CD19 and CD90 microbeads was added during the bead incubation step. Flowthrough was centrifuged for 7 min at 500g and resuspended in PBS with 0.04% BSA for samples that were to be sent directly for scRNA-seq. For each sample the single-cell suspension was loaded onto Chromium, aiming for 6,000 cells. Libraries were prepared according to the 10X Genomics Chromium Single Cell 3′ Reagent Kits User Guide (v.3 Chemistry). In each experiment, both WT and Akita mice were sequenced to avoid confounding from batch effect. Libraries were sequenced using either an Illumina Nextseq 550 or a Novaseq 6000. On Nextseq, libraries were sequenced according to the guidelines except that 56 base pairs were sequenced on R2 and 89 on Novaseq R2.

For scRNA-seq experiments with cell hashing, cells were incubated for 5 min with 1 µg ml−1 anti-CD16/32 (BioLegend) to block Fc receptors before the addition of the antibody mix including hashing antibodies. Cells were incubated for 30 min on ice, 10 ml of PBS was added and cells were then centrifuged for 7 min at 500g. The supernatant was removed and cells were resuspended in PBS, filtered through a 40 µm mesh and sorted by flow cytometry as described above. Sorted cells were centrifuged for 7 min at 500g, resuspended in PBS with 0.04% BSA and counted before being sent for scRNA-seq. For each sample, the single-cell suspension was loaded onto Chromium and libraries prepared as described above. Using hashing antibodies (BioLegend TotalSeq B0303, B0304, B0305 and B0306), we pooled two control and two STZ-treated mice per sample. Sequencing libraries for transcriptomes and hashtags were prepared according to the 10X Genomics Chromium Single Cell 3′ Reagent Kits User Guide with Feature Barcoding Technology (v.3.1 Chemistry). Subsequently, libraries were sequenced on a Novaseq 6000, with 89 base pair R2.

For analysis, bcl files were demultiplexed and converted to fastq files using bcl2fastq v.2.20.0.422 (mkfastq function in the CellRanger pipeline v.6.0.0). In experiments with Akita mice, reads from Novaseq were trimmed to 56 base pairs with trimmomatic v.0.36 that was sequenced on Nextseq to ensure comparability between samples. Subsequently, reads were mapped to the mouse genome (mm10) combined with the influenza A genome (Puerto Rico 1934, H1N1) and gene expression was quantified using the unique molecular identifier count function in CellRanger pipeline v.6.0.0). For Akita experiments, 26 samples (four day 0 Akita, four day 0 WT, five day 1 Akita, five day 1 WT, four day 10 Akita and four day 10 WT) were then aggregated with the aggr function in the CellRanger pipeline. Cell number for specific populations was obtained by multiplying cell type frequency from single-cell data by total lung cell count. For STZ, two single-cell libraries with four hashed samples in each were also aggregated.

Analysis of whole-lung single-cell transcriptomes

First, cells with fewer than 600 detected transcripts, fewer than 200 detected genes and over 20% mitochondrial reads were removed. Doublets were identified by finding clusters of cells displaying gene expression patterns of two cell types simultaneously, and were removed. Data were then normalized using the LogNormalize method from NormalizeData function, and 2,000 highly variable genes were identified using the ‘vst’ method. We calculated principal component analysis using highly variable genes, and k-nearest neighbour for the dataset was computed using the first 30 principal components. Finally, cells were clustered using a shared nearest-neighbour modularity optimization-based clustering algorithm. All functions mentioned above are part of the Seurat v.4.0.1 package in R34,35. To achieve better clustering results, we used a stepwise approach and divided the obtained clusters into five datasets based on their markers, containing immune cells, epithelial cells, endothelial cells, stromal cells and cells expressing high levels of cell cycle genes. For each of the subsets, variable genes, principal component analysis, clustering and UMAP were recalculated. Following reclustering—because immune cells featured a significant heterogeneity—we decided to further subdivide into T cells, B cells, mononuclear phagocytes and the remainder immune cell fraction (neutrophils and basophils). Because we were particularly interested in DC, mononuclear phagocytes were subdivided again into monocytes/macrophages and DC. All of these clusters were annotated using the Immgen database[36] and previously published data36,37,38,39. For analysis, cells with fewer than 300 detected genes and over 20% mitochondrial reads were removed. Contaminant cells (mostly NK and B cells) were identified by finding clusters of cells expressing gene expression patterns of cell types other than DC and were then removed. Quantified hashtags were used to demultiplex samples—we considered a hashtag as a single if particular barcode abundance was higher than five times that of the second-most abundant. Cells for which hashtags were not identified, or if more than one hashtag was found, were removed. Data were then normalized, scaled and clustered as described above for Akita mice. We used our single-cell transcriptomic data from the whole lung as a reference for cell type annotation. Differential expression analysis between conditions was performed on pseudobulk counts for each cell type in each sample using DESeq2 (ref. [40]). Functional analysis of differentially expressed genes, ordered according to adjusted P values, was performed using g:Profiler2 with default settings and multiple hypothesis testing adjustment using all mouse genes as background control[41]. The log_10 false discovery rate-adjusted P values were plotted as barplots. For heatmaps, Gene Ontology lists were obtained from Ensembl BioMart. KEGG pathway enrichment analysis was performed using all differentially expressed genes higher in PBS controls than in STZ-treated animals, with the gProfiler2 R package using standard settings.

CUT&Tag chromatin profiling

H3K27 trimethylation (H3K27me3) and H3K27ac were profiled from sorted DC using a modified CUT&RUN protocol[30]. Nuclei were extracted from 200,000 sorted cells. Cells were collected by centrifugation at 500g for 5 min at 4 °C and gently resuspended in 50 µl of lysis buffer containing 10 mM Tris pH 7.4, 10 mM NaCl, 3 mM MgCl_2, 1% BSA, 0.1% Tween-20, 0.1% IGEPAL, 0.01% Digitonin (Promega), 1 mM DTT and 1× complete protease inhibitor with incubation on ice for 3 min. Wash buffer (50  µl) containing 10 mM Tris pH 7.4, 10 mM NaCl, 3 mM MgCl_2, 1% BSA, 0.1% Tween-20, 1 mM DTT and 1× complete protease inhibitor was added. Nuclei were collected by centrifugation of 500g for 5 min at 4 °C and resuspended in 100 µl of room temperature NE Buffer. Concanavalin A beads (5  µl per sample; Epicypher) were activated by washing beads twice with 100 µl 10 µl−1 cold Bead Activation Buffer (20 mM HEPES pH 7.9, 10 mM KCl, 1 mM CaCl_2, 1 mM MnCl_2). Beads were resuspended in 10 µl per sample cold Bead Activation Buffer. Nuclei were incubated with concanavalin beads at room temperature for 10 min. Beads were placed on the magnet, supernatant was removed and nuclei were resuspended in 50 µl of cold wash buffer (20 mM HEPES pH 7.5, 150 mM NaCl, 0.5 mM spermidine, 1× complete protease inhibitor, 2 mM EDTA). Primary antibody (0.5  µl of H3K27me3 antibody, no. C36B11, Cell Signaling; H3K27ac antibody, no. D5E4, Cell Signaling; Isotype control antibody, no. DA1E, Cell Signaling) was added to each sample and incubated overnight at the nutator at 4 °C. Beads were placed on the magnet, supernatant was removed and nuclei were resuspended in 50 µl of digitonin buffer (20 mM HEPES pH 7.5, 150 mM NaCl, 0.5 mM spermidine, 1×  complete protease inhibitor, 0.01% digitonin, 2 mM EDTA) and 0.5 µg of anti-rabbit secondary antibody, Epicypher was added, mixed and incubated for 30 min at room temperature. While beads were on the magnet, they were washed twice with 200 µl of cold digitonin buffer and then resuspended in 50 µl of high-salt digitonin buffer (20 mM HEPES pH 7.5, 300 mM NaCl, 0.5 mM spermidine, 1× complete protease inhibitor, 0.01% digitonin, 2 mM EDTA); 2.5 µl of pAG-Tn5 (Epicypher) was then added to each sample, followed by vortexing and incubation for 1 h at room temperature. Nuclei were washed twice with high-salt digitonin buffer and 50 µl of cold tagmentation buffer was added (20 mM HEPES pH 7.5, 300 mM NaCl, 10 mM MgCl_2, 0.5 mM spermidine, 1× complete protease inhibitor) followed by incubation for 1 h at 37 °C. Following tagmentation, nuclei were washed with 50 µl of 10 mM N-[tris(hydroxymethyl)methyl]-3-aminopropanesulfonic acid (TAPS) pH 8.5 and 0.2 mM EDTA. To release DNA fragments, 5 µl of 10 mM TAPS pH 8.5 with 0.1% SDS was added to nuclei with incubation for 1 h at 58 °C. Subsequently, 15 µl of 0.67% Triton-X was added to neutralize SDS. Into 20 µl of sample, 25 µl of non-hot-start CUTANA High Fidelity 2× PCR Master Mix and 2 µl each of primers i5 and i7 (Illumina) were added with incubation at 58 °C for 5 min, 72 °C for 5 min, 98 °C for 45 min and then 18 cycles of 15 s at 98 °C and 10 s at 60 °C, followed by a final extension at 72 °C for 1 min. Libraries were cleaned up using 1.3× AMPure beads as per the manufacturer’s recommendations and eluted in 15 µl of TE buffer.

For analysis, bcl files were demultiplexed and converted to fastq files with bcl2fastq v.2.20.0.422. Subsequently, reads were trimmed to remove adaptors using fastp v.0.23.0 with standard parameters. Mapping to the GRCm38 genome was performed using bowtie2 v.2.3.5.1 and the following parameters: --local --very-sensitive-local --no-unal --no-mixed --no-discordant --phred33 -I 10 -X 700, and deduplicated with picard v.2.22.8 (ref. [42]). Files were converted with SAMtools v.1.9 and BEDtools v.2.26.0 to generate bedgraph files[43]. Peaks were called with SEACR v.1.3 using isotype control data as background, thereby identifying genes closest to peaks[44]. Data were subsampled to have the same coverage across samples, and reads in peaks were counted with BEDtools multicov[43]. To find differentially abundant peaks we used DESeq2 (ref. [40]).

DC T cell coculture

cDC1 and cDC2 were sorted from naïve lungs of WT or Akita mice. CD8+ T cells were obtained from splenocytes of T cell receptor-transgenic OT-I mice, in which all CD8+ T cells recognize the SIINFEKL peptide of OVA. Isolation was performed using CD90.2 MACS-bead (Miltenyi) pre-enrichment and subsequent flow cytometry-based sorting of CD8+CD11c− aufluorescent-negative cells. T cells and DC were then cocultured for 4 days in complete IMDM medium (Life Technologies) with 100 mg ml−1 OVA protein.

Model of allergic airway inflammation

Mice received 10 μg of HDM in 50 µl of PBS (Citeq) on day 0 for sensitization, and were then challenged daily with 10 μg of HDM on days 7–11 with analysis on day 14.

Transfer of HDM-pulsed lung DC

Mice were intratracheally administered 200 μg of HDM 24 h post infection. Lung DC were sorted and intratracheally transferred to naïve WT recipients. Each mouse received 5 × 10^5 cells. Animals were then challenged daily with 10 μg of HDM on days 7–11, analysis performed on day 14.

Transfer of UV-PR8-pulsed lung DC

PR8 influenza virus was inactivated under an ultraviolet lamp for 30 min. Mice were intratracheally administered 2.5 × 10^6 plaque-forming units (pfu) inactivated PR8 20 h post infection. Lung DC sorted and transferred to naïve recipients with 3 × 10^5 cells per recipient. After 10 days, animals were infected with 500 pfu PR8 and euthanized 7 days post infection.

DC-mediated antigen uptake and transport

Animals intratracheally received 200 μg of Ovalbumin-AlexaFluor647 (Invitrogen) and either 100 μg of HDM (Citeq) or 500 pfu PR8. Lungs and dLN were collected at the indicated time points.

Bone marrow chimeras

WT recipients were sublethally irradiated with 9.5 Gy and transplanted with Zbtb46-DTR bone marrow 1 day later. Animals were used for experiments 8 weeks following bone marrow reconstitution. In relevant groups, animals received 20 mg per kg−1 bodyweight diphtheria toxin every other day for 10 days.

Free fluid measurement

Free fluid was measured using nuclear magnetic resonance with a Bruker minispec LF50/mq7.5 MHz live mouse analyser.

Histology

Lungs were removed and fixed in 4% formaldehyde. Tissues were processed and stained with haematoxylin and eosin. Lungs were evaluated in a blind manner by a certified pathologist scoring according to the severity of inflammation (inflammatory score).

Immunoblotting

Cells were lysed in RIPA buffer containing protease and phosphatase inhibitors, incubated for 30 min at 4 °C and centrifuged at 15,000g for 10 min at 4 °C. Samples were run on 12% acrylamide gels and transferred to nitrocellulose membranes. Immunoblot analysis was performed using anti‐H3K27ac polyclonal antibody (no. ab4729, 1:1,000; Abcam), beta-actin (no. MA5-15739, 1:1,000, Thermo), goat anti-mouse HRP (no. 115-035-205, 1:5,000, Jackson Laboratories) and goat anti-rabbit HRP (no. 111-035-003, 1:5,000, Jackson Laboratories). Immunoblot imaging and band intensity quantification were performed using the Gel Doc XR+ system (Bio-Rad).

RNA isolation and quantitative PCR

Lung lobes were collected in Trizol (Bio-Lab), frozen on dry ice and stored at −80 °C until required for RNA isolation. To isolate RNA, samples were thawed, a metal bead was added and cells were lysed in a tissue lyser (Qiagen) at 30 Hz for 2 min. Next, 200 µl of chloroform was added and samples were vortexed and centrifuged at 13,000 rpm for 15 min at 4 °C. The clear layer was transferred to a fresh tube, 500 µl of isopropanol was added and the sample was precipitated at −20 °C overnight. Samples were centrifuged for 8 min at 10,000 rpm at 4 °C and pellets were washed once in ethanol and resuspended in 50 µl of DNase digestion mix (Sigma). Samples were treated according to the manufacturer’s instructions. Subsequently 1 µg of RNA was reverse transcribed using a high-capacity reverse transcription kit (Applied Biosystems). For quantitative PCR of host and viral genes, DNA templates were diluted to obtain 1 µg per reaction. Amplifications were performed with the following primer sets: PR8 forward, 5′-AGATGAGTCTTCTAACCGAGGTCG-3′; PR8 reverse, 5′-TGCAAAAACATCTTCAAGTCTCTG-3′; PVM forward, 5′-AGGACTCTGCCAGATGGTTG-3′; PVM reverse, 5′ CAGGGAAACTCAAAGGGTCA-3′; HPRT forward, 5′-TCAGTCAACGGGGGACATAAA-3′; HPRT reverse, 5′-GGGGCTGTACTGCTTAACCAG-3′; Ifnb1 forward, 5′-TCCGAGCAGAGATCTTCAGGAA-3′; Ifnb1 reverse, 5′-TGCAACCACCACTCATTCTGAG-3′. Fast SYBR Green Master Mix (Thermo Fisher Scientific) was used in duplicates. Amplification conditions were as follows: denaturation at 95 °C for 20 s, followed by 40 cycles of denaturation 95 °C for 1 s; annealing at 60 °C for 20 s followed by the melting curve. Data were analysed using the ∆Ct method.

Targeted metabolomics

Acetyl-CoA

Cells were sorted into 100% methanol to give a final concentration of 60% methanol and frozen in liquid nitrogen. The buffer was evaporated using speedvac and stored at −80 °C until further processing. For liquid chromatography–tandem mass spectrometry (LC–MS/MS), samples were thawed on ice and ultrasonicated at 4 °C and 13,000g for 10 min. Supernatants were transferred to new tubes, lyophilized and reconstituted in LC–MS-grade water. Chromatography was performed using a Shimadzu UHPLC System on an Xselect column (HSS T3, 3.5 µm particle size, 100 Å pore size, 100 × 2.1 mm^2). Injection volume was 20 μl, oven temperature was maintained at 40 °C and autosampler temperature was maintained at 5 °C. Chromatographic separation was achieved using a linear gradient programme at a constant flow rate (350 µl min−1) over a total run time of 7 min, from 83 to 5% solvent A (10 mM ammonium acetate in water pH 9.0). Methanol:water (1:1) was used for washing the needle before each injection cycle. All samples were analysed in duplicate. Acetyl-CoA was detected using an AB Sciex Triple Quad 5500 mass spectrometer in negative-ion mode, with electrospray ionization and multiple-reaction monitoring mode of acquisition. The IonDriveTM Turbo V source temperature was set at 450 °C, with ion spray voltage at 5,000 V. The curtain gas was set at 30.0 psi. The nebulizer gas (gas 1) was set to 50 psi, the turbo heater gas (gas 2) was set to 50 psi, the collision gas was set to high and dwell time was 20 ms. Two transitions were monitored: m/z 303.1 (quantifier) and m/z 428 (qualifier). Data acquisition was performed using Analyst 1.7.1, and data were analysed with Sciex OS Software.

Pyruvate

Cells were lysed and extracted twice with 100 µl of 80% aqueous methanol. Insoluble material was pelleted in a centrifuge at 4 °C. The supernatant was collected and evaporated. The residue was resuspended in 100 µl of water and used for consequent LC–MS/MS analyses through derivatization, as previously described[45]. In brief, 100 µl of standard or sample was mixed with 50 μl of 35 mM 3-nitrophenylhydrazine (Sigma) in 75% methanol, 50 μl of 105 mM N-(3-dimethylaminopropyl)-N′-ethylcarbodiimide (Sigma) in methanol and 50 μl of 2.5% pyridine in methanol. The reaction mixture was shaken at 4 °C for 30 min and then evaporated. The residue was resuspended in 50 μl of 100% methanol, filtered and used for LC–MS/MS analysis. The LC–MS/MS instrument comprised an Acquity I-class UPLC system (Waters) and Xevo TQ-S triple quadrupole mass spectrometer (Waters), equipped with an electrospray ion source and operated in negative-ion mode. MassLynx and TargetLynx software (v.4.2, Waters) were applied for the acquisition and analysis of data. Chromatographic separation was performed on a 150 × 2.1-mm^2-internal diameter, 1.8 μm UPLC HSS T3 column (Waters Acquity) with mobile phases A (0.03% aqueous formic acid) and B (0.03% formic acid in acetonitrile) at a flow rate of 0.3 ml min−1 and column temperature of 40 °C. The gradient used was as follows: for the first 1.5 min, the column was held at 5% B, then to 35 min a linear increase to 73% B and for the following 30 s to 90% B, then to 35.5–38.0 min back to 5% B and equilibration at 5% B for 7 min. Samples maintained at 4 °C were automatically injected in a volume of 5 μl. For MS, argon was used as the collision gas at a flow of 0.10 ml min−1. Capillary voltage was set to 2.23 kV, source temperature to 150 °C, desolvation temperature to 400 °C, cone gas flow to 150 l h−1 and desolvation gas flow to 800 l h−1. Multiple-reaction monitoring transitions for pyruvate were as follows: 357.2 (12C), 358.2 (13C_1), 359.2 (13C_2) and 360.2 (13C_3) m/z for parent ions and 136.9 m/z for the fragment ion, with collision energy 15 eV.

Insulin supplementation

Osmotic minipumps (Alzet, model 2004) were used (infusing the compound at a rate of 0.25 µl h−1 for 28 days). The pumps were filled with 200 µl of human insulin (Sigma) diluted in PBS (-Ca2+, -Mg2+). Vehicle control pumps contained an equivalent volume of PBS (-Ca2+, -Mg2+). Mice were anaesthetized by intraperitoneal injection of ketamine (100 mg kg−1) and xylazine (10 mg kg−1). The skin of the neck was shaved and disinfected with 70% ethanol. An incision was made in the skin and osmotic pumps subcutaneously inserted following minimal dissection and placed above the right hind flank. The cut was closed with sterile surgical clips and mice were frequently monitored for any signs of stress, bleeding, pain, discharge or abnormal behaviour.

ANA treatment

Mice received 5 mg kg−1 ANA (Sigma) in 100 µl of corn oil (Sigma) intraperitoneally daily for the indicated durations.

2-NBDG treatment

Mice received 100 μl of 5 mM 2-NBDG (Thermo Fisher Scientific) intravenously and euthanized after 30 min.

2-DG treatment

Mice received 240 mg kg−1 2-DG (Sigma) in 200 µl of PBS intraperitoneally for the indicated durations.

Plasma parameters

Blood samples were centrifuged for 15 min at 4 °C and 12,000g to separate plasma from cells. Enzyme activity and molecule concentrations were measured using a Roche Cobas 111 serum analyser according to the manufacturer’s instructions.

Insulin measurement

Plasma insulin was measured using the Ultrasensitive Mouse Insulin ELISA kit (Crystal Chem) according to the manufacturer’s instructions.

Statistical analysis and reproducibility

Statistical tests were performed using GraphPad Prism 9.2 and R. For pooled analysis of results from different independent repeats, all mice from the same experimental group were pooled and a new statistical comparison was made for the entire pooled experiment, as performed for individual repeats. All measurements were taken from distinct samples. For all datasets, normality was calculated using the Shapiro–Wilk test and parametric or non-parametric tests were used accordingly. For comparisons between two groups with normal distribution, a two-sided unpaired t-test was performed; for comparisons between two groups with non-normal distribution, a two-sided Mann–Whitney U-test was used; for comparison of more than two groups, ANOVA was used with correction for multiple comparisons using Holm–Sidak for parametric-distributed datasets and the Kruskal–Wallis test with Dunn’s correction for non-parametric-distributed datasets. All exact P values are presented in Supplementary Table 1. P < 0.05 was considered significant; *P < 0.05, **P < 0.01, ***P < 0.001, ****P < 0.0001. All experiments were repeated between two and seven times.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

All raw sequencing data are deposited to Array Express with the following accession numbers: scRNA-seq raw data from WT and Akita mice over the course of infection, E-MTAB-11394; hashed scRNA-seq data of DC from STZ model, E-MTAB-11393; CUT&RUN data, E-MTAB-11390.

References

1. Erener, S. Diabetes, infection risk and COVID-19. Mol. Metab. 39, 101044 (2020).

2. Mobasseri, M. et al. Prevalence and incidence of type 1 diabetes in the world: a systematic review and meta-analysis. Health Promot. Perspect. 10, 98–115 (2020).

3. Katsarou, A. et al. Type 1 diabetes mellitus. Nat. Rev. Dis. Primers 3, 17016 (2017).

4. Hu, Y. et al. Prevalence and severity of corona virus disease 2019 (COVID-19): a systematic review and meta-analysis. J. Clin. Virol. 127, 104371 (2020).

5. Barron, E. et al. Associations of type 1 and type 2 diabetes with COVID-19-related mortality in England: a whole-population study. Lancet Diabetes Endocrinol. 8, 813–822 (2020).

6. Ortega, E. et al. Risk factors for severe outcomes in people with diabetes hospitalised for COVID-19: a cross-sectional database study. BMJ Open 11, e051237 (2021).

7. Brosh-Nissimov, T. et al. BNT162b2 vaccine breakthrough: clinical characteristics of 152 fully vaccinated hospitalized COVID-19 patients in Israel. Clin. Microbiol. Infect. https://doi.org/10.1016/j.cmi.2021.06.036 (2021).

8. Huo, C. et al. Mice with type 1 diabetes exhibit increased susceptibility to influenza A virus. Microb. Pathog. 113, 233–241 (2017).

9. Knapp, S. Diabetes and infection: is there a link?--A mini-review. Gerontology 59, 99–104, 2013.

10. Bosteels, C. et al. Inflammatory type 2 cDCs acquire features of cDC1s and macrophages to orchestrate immunity to respiratory virus infection. Immunity 52, 1039–1056 (2020).

11. Nobs, S. P. et al. PPARγ in dendritic cells and T cells drives pathogenic type-2 effector responses in lung inflammation. J. Exp. Med. 214, 3015–3035 (2017).

12. Saragovi, A. et al. Systemic hypoxia inhibits T cell response by limiting mitobiogenesis via matrix substrate-level phosphorylation arrest. eLife https://doi.org/10.7554/eLife.56612 (2020).

13. Moussaieff, A. et al. Glycolysis-mediated changes in acetyl-CoA and histone acetylation control the early differentiation of embryonic stem cells. Cell Metab. 21, 392–402 (2015).

14. Zhang, S., Hulver, M. W., McMillan, R. P., Cline, M. A. & Gilbert, E. R. The pivotal role of pyruvate dehydrogenase kinases in metabolic flexibility. Nutr. Metab. (Lond.) 11, 10 (2014).

15. Geltink, R. I. K., Kyle, R. L. & Pearce, E. L. Unraveling the complex interplay between T cell metabolism and function. Annu. Rev. Immunol. 36, 461–488 (2018).

16. Ryan, D. G. & O’Neill, L. A. J. Krebs cycle reborn in macrophage immunometabolism. Annu. Rev. Immunol. 38, 289–313 (2020).

17. Wang, A. et al. Opposing effects of fasting metabolism on tissue tolerance in bacterial and viral inflammation. Cell 166, 1512–1525 (2016).

18. Thomas, A. M. et al. Brief exposure to hyperglycemia activates dendritic cells in vitro and in vivo. J. Cell. Physiol. 235, 5120–5129 (2020).

19. Pearce, E. J. & Everts, B. Dendritic cell metabolism. Nat. Rev. Immunol. 15, 18–29 (2015).

20. Krawczyk, C. M. et al. Toll-like receptor-induced changes in glycolytic metabolism regulate dendritic cell activation. Blood 115, 4742–4749 (2010).

21. Wu, D. et al. Type 1 interferons induce changes in core metabolism that are critical for immune function. Immunity 44, 1325–1336 (2016).

22. Kinnaird, A., Zhao, S., Wellen, K. E. & Michelakis, E. D. Metabolic control of epigenetics in cancer. Nat. Rev. Cancer 16, 694–707 (2016).

23. Shvedunova, M. & Akhtar, A. Modulation of cellular processes by histone and non-histone protein acetylation. Nat. Rev. Mol. Cell Biol. 23, 329–349 (2022).

24. Zhan, Y. et al. Differential requirement for the Polycomb repressor complex 2 in dendritic cell and tissue-resident myeloid cell homeostasis. Sci. Immunol. 6, eabf7268 (2021).

25. Lavarone, E., Barbieri, C. M. & Pasini, D. Dissecting the role of H3K27 acetylation and methylation in PRC2 mediated control of cellular identity. Nat. Commun. 10, 1679 (2019).

26. Thwe, P. M. et al. Cell-intrinsic glycogen metabolism supports early glycolytic reprogramming required for dendritic cell immune responses. Cell Metab. 30, 225 (2019).

27. Cheng, Y. et al. Targeting epigenetic regulators for cancer therapy: mechanisms and advances in clinical trials. Signal Transduct. Target. Ther. 4, 62 (2019).

28. Leisman, D. E. et al. Cytokine elevation in severe and critical COVID-19: a rapid systematic review, meta-analysis, and comparison with other inflammatory syndromes. Lancet Respir. Med. 8, 1233–1244 (2020).

29. Yoshioka, M., Kayo, T., Ikeda, T. & Koizumi, A. A novel locus, Mody4, distal to D7Mit189 on chromosome 7 determines early-onset NIDDM in nonobese C57BL/6 (Akita) mutant mice. Diabetes 46, 887–894 (1997).

30. Skene, P. J. & Henikoff, S. An efficient targeted nuclease strategy for high-resolution mapping of DNA binding sites. eLife https://doi.org/10.7554/eLife.21856 (2017).

31. Thaiss, C. A. et al. Hyperglycemia drives intestinal barrier dysfunction and risk for enteric infection. Science 359, 1376–1383 (2018).

32. Jenkins, M. R., Webby, R., Doherty, P. C. & Turner, S. J. Addition of a prominent epitope affects influenza A virus-specific CD8+ T cell immunodominance hierarchies when antigen is limiting. J. Immunol. 177, 2917–2925 (2006).

33. Millard, P. et al. IsoCor: isotope correction for high-resolution MS labeling experiments. Bioinformatics 35, 4484–4487 (2019).

34. Stuart, T. et al. Comprehensive integration of single-cell data. Cell 177, 1888–1902 (2019).

35. Hao, Y. et al. Integrated analysis of multimodal single-cell data. Cell 184, 3573–3587 (2021).

36. Shay, T. & Kang, J. Immunological Genome Project and systems immunology. Trends Immunol. 34, 602–609 (2013).

37. Zilionis, R. et al. Single-cell transcriptomics of human and mouse lung cancers reveals conserved myeloid populations across individuals and species. Immunity 50, 1317–1334 (2019).

38. Habermann, A. C. et al. Single-cell RNA sequencing reveals profibrotic roles of distinct epithelial and mesenchymal lineages in pulmonary fibrosis. Sci. Adv. 6, eaba1972 (2020).

39. Aran, D. et al. Reference-based analysis of lung single-cell sequencing reveals a transitional profibrotic macrophage. Nat. Immunol. 20, 163–172 (2019).

40. Love, M. I., Huber, W. & Anders, S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol. 15, 550 (2014).

41. Raudvere, U. et al. g:Profiler: a web server for functional enrichment analysis and conversions of gene lists (2019 update). Nucleic Acids Res. 47, W191–W198 (2019).

42. Langmead, B. & Salzberg, S. L. Fast gapped-read alignment with Bowtie 2. Nat. Methods 9, 357–359 (2012).

43. Quinlan, A. R. & Hall, I. M. BEDTools: a flexible suite of utilities for comparing genomic features. Bioinformatics 26, 841–842 (2010).

44. Meers, M. P., Tenenbaum, D. & Henikoff, S. Peak calling by Sparse Enrichment Analysis for CUT&RUN chromatin profiling. Epigenetics Chromatin 12, 42 (2019).

45. Meng, X. et al. Simultaneous 3-nitrophenylhydrazine derivatization strategy of carbonyl, carboxyl and phosphoryl submetabolome for LC-MS/MS-based targeted metabolomics with improved sensitivity and coverage. Anal. Chem. 93, 10075–10083 (2021).

Acknowledgements

We thank the members of the Elinav lab, Weizmann Institute of Science, and members of the microbiome and cancer division, German Cancer Research Center (DKFZ), for insightful discussions. We thank the EFSD/Lilly European Diabetes Research Programme for supporting key aspects of this research. We thank E. Selzer for providing Pdk2/3/4-deficient mice; Y. Merbl for providing OT-I and OT-II mice; I. Sagi for providing PR8 influenza virus; and N. Garbi and R. Alon for providing OVA(SIINFEKL)-PR8 virus. Schemes in Supplementary Fig. 12 were created with a licensed version of BioRender. S.P.N was supported by an SNSF Postdoc Mobility fellowship, SNSF Early Postdoc Mobility Fellowship, EMBO Long-term Fellowship no. 2017-767 and a fellowship from the Swiss Friends of the Weizmann Institute of Science. A.A.K. is a recipient of EMBO Long-term Fellowship no. 2016‐1088 and the European Union’s Horizon 2020 research and innovation programme under Marie Sklodowska‐Curie grant agreement no. 747114. S.K.A. is supported by the Israeli Ministry of Science and Technology Zvi Yanai Fellowship. I.S. is supported by DFG (no. SP1902/1-1). H.S. is the incumbent of the Vera Rosenberg Schwartz Research Fellow Chair. L.S., J.P. and E.E. are supported by a grant from the Hector foundation. E.E. is supported by the Leona M. and Harry B. Helmsley Charitable Trust, Adelis Foundation, Pearl Welinsky Merlo Scientific Progress Research Fund, Park Avenue Charitable Fund, Hanna and Dr Ludwik Wallach Cancer Research Fund, Daniel Morris Trust, Wolfson Family Charitable Trust and Wolfson Foundation, Ben B. and Joyce E. Eisenberg Foundation, White Rose International Foundation, Estate of Malka Moskowitz, Estate of Myron H. Ackerman, Estate of Bernard Bishin for the WIS-Clalit Program, Else Kröener-Fresenius Foundation, Jeanne and Joseph Nissim Center for Life Sciences Research, A. Moussaieff, M. de Botton, Vainboim family, A. Davidoff, the V. R. Schwartz Research Fellow Chair and by grants funded by the European Research Council, European Union Horizon Programme, Israel Science Foundation, Israel Ministry of Science and Technology, Israel Ministry of Health, Helmholtz Foundation, Garvan Institute of Medical Research, European Crohn’s and Colitis Organization, Kenneth Rainin Foundation, Charlie Teo Foundation, CueALZ, Lupus Research Alliance, Jose Carreras Foundation, Mark Foundation for Cancer Research, Deutsch-Israelische Projektkooperation, IDSA Foundation and Wellcome Trust. E.E. is the incumbent of the Sir Marc and Lady Tania Feldmann Professorial Chair, a senior fellow of the Canadian Institute of Advanced Research (CIFAR) and an international scholar of the Bill & Melinda Gates Foundation and Howard Hughes Medical Institute (HHMI).

Funding

Open access funding provided by Deutsches Krebsforschungszentrum (DKFZ).

Author information

Author notes

1. Aleksandra A. Kolodziejczyk

Present address: International Institute of Molecular and Cellular Biology, Warsaw, Poland

2. These authors contributed equally: Samuel Philip Nobs, Aleksandra A. Kolodziejczyk

Authors and Affiliations

1. Systems Immunology Department, Weizmann Institute of Science, Rehovot, Israel

Samuel Philip Nobs, Aleksandra A. Kolodziejczyk, Nir Horesh, Christine Botscharnikow, Ella Herzog, Gayatree Mohapatra, Sophia Hejndorf, Ryan-James Hodgetts, Igor Spivak, Leviel Fluhr, Denise Kviatcovsky, Mally Dori-Bachash, Yiming He, Sara Ferrini, Hagit Shapiro, Suhaib K. Abdeen & Eran Elinav

2. Department of Molecular Cell Biology, Weizmann Institute of Science, Rehovot, Israel

Lital Adler & Ayelet Erez

3. Department of General Surgery and Transplantations, Sheba Medical Center, Ramat Gan, Israel

Nir Horesh

4. Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel

Nir Horesh

5. Division of Microbiome & Cancer, DKFZ, Heidelberg, Germany

Lena Schorr, Jens Puschhof & Eran Elinav

6. Faculty of Biosciences, Heidelberg University, Heidelberg, Germany

Lena Schorr

7. The Institute for Drug Research, Hebrew University of Jerusalem, Jerusalem, Israel

Anish Zacharia, Suzanne Njuki, Dinorah Barasch & Arieh Moussaieff

8. Department of Veterinary Resources, Weizmann Institute of Science, Rehovot, Israel

Noa Stettner & Alon Harmelin

9. Department of Biological Services, Weizmann Institute of Science, Rehovot, Israel

Alexander Brandis & Tevie Mehlman

10. Institute of Molecular Health Sciences, ETH Zurich, Zurich, Switzerland

Manfred Kopf

Contributions

S.P.N. and A.A.K. conceived the study, designed, performed, analysed and interpreted the experiments and wrote the manuscript. A.A.K. designed and headed all computational aspects related to this study. L.A., N.H., C.B., E.H., G.M., S.H., R.-J.H., I.S., L.S., L.F., D.K., A.Z., S.N., D.B., M.D.-B. Y.H., S.F. and J.P. contributed to experiments. H.S. and N.S. coordinated key animal experimentation. A.H. scored histopathologies. A.B. and T.M. performed and analysed metabolomics experiments. M.K. provided critical feedback and reagents. A.E. and A.M. helped conceptualize and supervise multiple metabolic aspects of the study. S.K.A. cosupervised the study and wrote the manuscript. E.E. conceived the study, supervised participants, interpreted the experiments and wrote the manuscript.

Corresponding authors

Correspondence to Suhaib K. Abdeen or Eran Elinav.

Ethics declarations

Competing interests

E.E. is a scienrtific cofounder of DayTwo and BiomX and an advisor to Purposebio, Igen, Aposense and Zoe in topics unrelated to this work. The remaining authors declare no competing interests.

Peer review

Peer review information

Nature thanks Rachel Perry, Alice Prince and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Diabetes-associated susceptibility to respiratory viral infection is associated with broadly impaired lung adaptive immunity.

a-m, WT (n = 16) and Akita mice (n = 7) infected with 50pfu PR8, analyzed at 10 d.p.i. a-b, Lung histology, two-sided Mann Whitney U-test. c, Lung CD4+ T cells, two-sided Mann Whitney U-test, lung CD8+ T cells, two-sided unpaired t-test. d, Lung Ki-67+CD8+ T cells, two-sided unpaired t-test. e, Lung IFNγ+CD8+ T cells, two-sided unpaired t-test. f, Lung T-bet+CD8+ T cells, two-sided Mann Whitney U-test. g, Lung FoxP3+CD4+ T cells, two-sided unpaired t-test. h, Lung B cells, two-sided Mann Whitney U-test. i, GC B cells, two-sided unpaired t-test. j-k, virus specific IgG2b, j, BAL, k, serum, two-sided Mann-Whitney U-test for EC50. l-m, virus-specific IgM, l, BAL m, serum, two-sided unpaired t-test for EC50. n-s, Mice infected with 50 pfu PR8, treated with insulin/PBS: WT+PBS (n = 16), WT+Ins (n = 15), Akita+PBS (n = 13), Akita+Ins (n = 15), pooled data from 2 experiments. n, Blood glucose. o, Lung CD4+ T cells, one-way ANOVA and Holm Sidak correction. p, Lung CD8+ T cells, Kruskal-Wallis and Dunn’s correction. q, Lung GC B cells, Kruskal-Wallis and Dunn’s correction. r-s, Virus-specific IgG2b, r, BAL, s, serum, Akita+PBS (n = 6), Akita+Ins (n = 9), one-way ANOVA with Tukey correction on EC50. All data mean+s.e.m. GC, Germinal centre.

Extended Data Fig. 2 Diabetes associated susceptibility to respiratory viral infection is associated with broadly impaired adaptive immunity in streptozotocin-induced diabetes.

a, Blood glucose, PBS (n = 20) or STZ (n = 16) administered mice, analyzed at 10 d.p.i., two-sided unpaired t-test. b, Survival, PBS (n = 10) or STZ (n = 10) administered mice, log-rank Mantel-Cox test. c-r, Mice infected with 50pfu PR8, administered PBS (n = 20) or STZ (n = 10 in e, n = 9 in all other panels), analyzed at 10 d.p.i. c, Lung Ifnβ1, two-sided Mann-Whitney U-test. d-e, Lung histology, d, representative sections, e, inflammatory score, two-sided Mann-Whitney U-test. f, Lung T cells, two-sided unpaired t-test. g-h, Flow cytometry,lung T-bet+CD8+ T cells, g, representative blots h, quantification, two-sided unpaired t-test. i, Lung IFNγ+CD8+ T cells, two-sided unpaired t-test. j, Lung Ki-67+CD8+ T cells, two-sided unpaired t-test. k, Lung FoxP3+CD4+ T cells, two-sided Mann-Whitney U-test. l, Lung B cells, two-sided Mann-Whitney U-test. m-n, Flow cytometry, GC B cells, m, representative blots, n, quantification, two-sided Mann-Whitney U-test. o-p, Virus specific IgG2b,o, BAL, p, serum, two-sided unpaired-test for EC50. q-r,Virus specific IgM, q, BAL, r, serum, two-sided Mann-Whitney test for EC50. All data mean+s.e.m.

Extended Data Fig. 3 Normalization of glucose levels in diabetic animals with insulin restores adaptive antiviral immunity and attenuates susceptibility to respiratory viral infection.

a-d, Mice infected with 50pfu PR8, administered STZ or PBS, treated with insulin/PBS: PBS+PBS (n = 19), PBS+Ins (n = 15), STZ+PBS (n = 4) STZ+Ins (n = 11), pooled data from 2 experiments. a, Survival curve (n = 11 mice/group), Log-rank Mantel Cox test. b, Lung CD4+ T cells, one-way ANOVA and Holm-Sidak correction. c, Lung CD8+ T cells, one-way ANOVA and Holm-Sidak correction. d, Lung GC B cells, Kruskall Wallis test and Dunn’s correction. e, Blood glucose, WT (n = 15) and Db/Db (n = 10) mice, two-sided unpaired t-test. f-j, WT (n = 14 for g, n = 15 for the other panels) and Db/Db (n = 10) mice infected with 50pfu PR8, pooled data from 3 experiments. f, Lung T cells, two-sided unpaired t-test for CD4+T cells and two-sided Mann-Whitney test for CD8+ T cells, respectively. g-h, Virus specific IgG2b, g, BAL, h, serum, two-sided Mann-Whitney test for EC50. i-j, IgM, i, BAL, j, serum, two-sided Mann-Whitney test for EC50. k, Blood insulin, WT (n = 15) and Db/Db (n = 10) mice, two-sided Mann-Whitney U-test. l, Lung Ifnβ1 expression, WT (n = 15) and Akita (n = 17) mice infected with 50pfu PVM, two-sided Mann-Whitney U-test. m-p, WT (n = 7) and Akita mice (n = 6 in n and n = 7 in the other panels) infected with 50pfu PVM. m, Lung T cells, two-sided unpaired t-test. n, Lung Ki-67+CD8+ T cells and T-bet+CD8+ T cells, two-sided Mann Whitney U-test. o, Lung B cells, two-sided Mann Whitney U-test. p, Lung GC B cells, two-sided Mann Whitney U-test. q-u, Mice infected with 50pfu PVM, administered STZ or PBS, treated with insulin/PBS: PBS+PBS (n = 14), PBS+Ins (n = 14), STZ+PBS (n = 13), STZ+Ins (n = 13), pooled data from 2 experiments. q, Lung viral RNA, Kruskall Wallis test and Dunn’s correction. r, Lung CD4+ T cells, one-way ANOVA and Holm-Sidak correction. s, Lung CD8+ T cells, Kruskall Wallis test and Dunn’s correction. t, Lung B cells, Kruskall Wallis test and Dunn’s correction. u, Lung GC B cells, Kruskall Wallis test and Dunn’s correction. All data mean+s.e.m.

Extended Data Fig. 4 Diabetes is associated with transcriptomic changes in respiratory viral infection.

a-h, scRNA-seq of lungs of WT and Akita mice (n = 4/group) collected during steady-state,1 d.p.i. and 10 d.p.i. with 50pfu PR8. Quantification of key cell subsets, two-sided Wilcoxon test. Boxplots show 25th to 75th percentiles, the 50th percentile denoted by a thicker line; whiskers show 1.5× interquartile range max. or min. if smaller than 1.5× interquartile range. i-j, Heatmap with z-score of gene expression of differentially expressed genes between cells from WT and Akita, 10 d.p.i (n = 4/group). Differentially expressed genes identified by DESeq2 on pseudobulk counts in each population and adjusted p-value < 0.05. DESeq2 is based on Negative Binomial GLM fitting and Wald statistics with Benjamini and Hochberg procedure. i, downregulated, and j, upregulated genes in Akita mice. AT1, type 1 alveolar epithelial cell; AT2, type 2 alveolar epithelial cell.

Extended Data Fig. 5 Hyperglycaemia dysregulates the lung DC compartment in infection and homeostasis.

a-d, Mice infected with 50pfu PR8, treated with Ins/PBS: WT+PBS (n = 6), WT+Ins (n = 7), Akita+PBS (n = 6), Akita+Ins (n = 9). a-c, one-way ANOVA and Holm-Sidak correction, d, Kruskal Wallis test with Dunn’s correction. a, Lung cDC2. b, Lung CD64+ DC. c, Lung IL-12+cDC1. d, Lung Ki-67+cDC1. e-f, Naïve WT (n = 8) and Akita (n = 8) mice, two-sided unpaired t-test. e, Lung cDC1, cDC2 and CD64+ DC. f, Lung Ki-67+ DC. g-i, Naïve WT (n = 10) and Akita (n = 10) mice, two-sided unpaired t-test. g, BM DC precursors. h, BM Pre-DC subsets. i, Blood lymphocytes. j-l, WT (n = 15) and Db/Db (n = 10) mice infected with 50pfu PR8, pooled data from 3 experiments. j, Lung cDC1, two-sided Mann Whitney U-test. k, Lung cDC2 and CD64+ DC, two-sided unpaired t-test. l, Lung Ki-67+ DC, two-sided unpaired t-test for cDC1 and CD64+ DC, two-sided Mann Whitney U-test for cDC2. m-p, WT and Akita mice infected with 50pfu PVM (n = 7 per group in m-o, WT (n = 8) and Akita (n = 10) mice in p). m, Lung cDC2, two-sided unpaired t-test and CD64+ DC, Mann Whitney U-test. n, Lung pDC, two-sided Mann Whitney U-test. o, Lung Ki-67+ DC, two-sided unpaired t-test. p, Lung IL-12+cDC1, two-sided unpaired t-test. q-r, WT and Akita mice treated with insulin/PBS: WT+PBS (n = 16), WT+Ins (n = 18), Akita+PBS (n = 14) Akita+Ins (n = 14), pooled data from 2 experiments. q, Lung cDC1, Kruskal Wallis test with Dunn’s correction.r, Lung cDC2 and CD64+ DC, Kruskal Wallis test with Dunn’s correction. s-t, WT and Akita mice treated with insulin/PBS: WT+PBS (n = 6), WT+Ins (n = 9), Akita+PBS (n = 7) Akita+Ins (n = 7), one-way ANOVA and Holm-Sidak. s, Lung Ki-67+cDC1. t, Lung IL-12+cDC1. All data mean+s.e.m. CDP, Common dendritic cell progenitor; MDP, Macrophage dendritic cell progenitor.

Extended Data Fig. 6 Insulin treatment improves outcomes of influenza infection.

a-e, Mice infected with 50pfu PR8, administered STZ or PBS, treated with insulin/PBS: PBS+PBS (n = 19), PBS+Ins (n = 15), STZ+PBS (n = 4), STZ+Ins (n = 11), pooled data from 2 experiments. a, Lung cDC1, Kruskal Wallis test with Dunn’s correction. b, Lung cDC2, one-way ANOVA and Holm-Sidak. c, Lung CD64+ DC, Kruskal Wallis test with Dunn’s correction. d, Lung Ki-67+cDC1, Kruskal Wallis test with Dunn’s correction. e, Lung IL-12+cDC1, one-way ANOVA and Holm-Sidak. f-h, Mice infected with 50pfu PVM, administered STZ or PBS, treated with insulin/PBS: PBS+PBS (n = 8), PBS+Ins (n = 7), STZ+PBS (n = 6), STZ+Ins (n = 8). f, Lung cDC1, Kruskall Wallis test and Dunn’s correction. g, Lung cDC2, Kruskall Wallis test and Dunn’s correction. h, Lung CD64+ DC, one-way ANOVA and Holm-Sidak correction. i-j, scRNA-seq of lung DC from PBS or STZ administered WT mice (n = 4/group). i, UMAP showing all DC populations. j, Balloon plots with key population markers. Size of dots represents percentage of cells in which the gene was detected, color corresponds to mean scaled gene expression of all cells in a cluster. k-p, Quantification of all lung DC subsets not shown in Fig. 2, two-sided Wilcoxon test. Boxplots show 25th to 75th percentiles, the 50th percentile denoted by a thicker line; whiskers show 1.5× interquartile range, or max. or min. if smaller than 1.5× interquartile range. All data mean+s.e.m.

Extended Data Fig. 7 Hyperglycaemia dysregulates the lung DC compartment in steady-state in the streptozotocin-induced diabetes model.

a-g, scRNA-seq of lung DC from PBS- or STZ-administered WT mice (n = 4/group). a-e, Heatmaps of differentially expressed genes with Z-score of tpm pseudobulk counts in a-b, CD64+ DC, c, cDC1, d, cDC2, and e, pDC. Differentially expressed genes identified by DESeq2 on pseudobulk counts in each population and adjusted p-value < 0.05. f, KEGG pathways reduced in pDC from STZ: bar plot showing KEGG pathway enrichment analysis with all differentially expressed genes higher in PBS controls than in STZ-administered mice, with gProfiler2 R package using standard settings; P values corrected for multiple hypothesis testing with g:SCS algorithm. g, KEGG pathways reduced in CD64+ DC, bar plot showing KEGG pathway enrichment analysis with all differentially expressed genes higher in PBS controls than in STZ-administered mice, with gProfiler2 R package using standard settings. P values corrected for multiple hypothesis testing using the g:SCS algorithm.

Extended Data Fig. 8 High glucose impairs lung DC immune functions.

a-d, WT (n = 9) and Akita mice (n = 6) infected with 50 pfu PR8 and intratracheally administered 100μg OVA-AF647. a, Representative flow cytometry, lung dLN OVA+DC. b, Percentage of lung dLN OVA+cDC2, two-sided unpaired t-test. c, Number of lung dLN OVA+DC, two-sided unpaired t-test. d, Frequency of lung OVA+DC, WT (n = 10) and Akita (n = 7) mice, cDC1 and CD64+ DC, two-sided unpaired t-test, cDC2, Mann Whitney U-test. e-g, Lung dLN cDC2 MFI, WT (n = 9) and Akita (n = 6) mice, two-sided unpaired t-test. e, CD40. f, CD80. g, CD86. h-j, Lung DC MFI, WT (n = 10) and Akita (n = 7) mice. h, CD40, two-sided Mann Whitney U-test. i, CD80, two-sided unpaired t-test. j, CD86, two-sided unpaired t-test. k, WT Lung DC incubated with high (50mM, n = 4) or normal (10mM, n = 4) glucose, cell viability, two-sided unpaired t-test. l-n, WT Lung cDC2 incubated with high (50mM, n = 4) or normal (10mM, n = 4) glucose, MFI, two-sided unpaired t-test. l, CD40. m, CD80. n, CD86. o-t, WT Lung DC incubated with 25 mM (n = 10) or 5 mM (n = 10) glucose, two-sided Mann Whitney U-test for p, two-sided unpaired t-test for all other panels. o, cDC1 CD40. p, cDC1 CD80. q, cDC1 CD86. r, cDC2 CD40. s, cDC2 CD80. t, cDC2 CD86. u-ab, WT and Akita mice intratracheally administered 100μg HDM + 100 μg OVA-AF647.u, Lung dLN OVA+DC, WT (n = 8) and Akita (n = 7) mice, two-sided unpaired t-test. v-x, Lung dLN DC MFI, WT (n = 10) and Akita (n = 5) mice, two-sided unpaired t-test. v, CD40. w, CD80. x, CD86. y, Lung OVA+DC frequency, WT (n = 10) and Akita (n = 7) mice, two-sided Mann Whitney U-test. z-ab, Lung DC MFI, WT (n = 9) and Akita (n = 5) mice. z, CD40, two-sided unpaired t-test. aa, CD80, two-sided Mann Whitney U-test. ab, CD86, two-sided Mann Whitney U-test. All data mean+s.e.m.

Extended Data Fig. 9 Hyperglycaemia protects from allergic lung inflammation.

a-c, WT and Akita mice intratracheally administered 10μg HDM at day 0, 7-11, analyzed at day 14. a, Lung eosinophils and neutrophils, WT (n = 5) and Akita (n = 6) mice, two-sided Mann Whitney U-test. b, Lung PAS staining and mucus quantification,WT (n = 13) and Akita (n = 9) mice, two-sided Mann Whitney U-test. c, Lung CD4+ T cells and CD8+ T cells, WT (n = 5) and Akita (n = 6) mice, two-sided unpaired t-test. d-e, STZ (n = 4)- or PBS (n = 9)-pretreated mice intratracheally received 10μg HDM at day 0, 7-11, analyzed on day 14. d, Lung eosinophils and neutrophils. e, Lung CD4+ T cells and CD8+ T cells. f, WT lung cDC2 incubated in vitro with high (50mM, n = 7) or normal (10mM, n = 7) glucose for 20 h, then co-cultured for 4 days with OT-I-CD8+ T cells in normal (10mM) glucose. CD8+T cells, two-sided Mann-Whitney U-test. g, WT lung cDC2 incubated in vitro with high (50mM, n = 12) or normal (10mM, n = 12) glucose for 20 h, then co-cultured for 4 days with OT-II-CD4+ T cells in normal (10mM) glucose. CD4+ T cells, two-sided unpaired t-test. h-i, T cell activation by anti-CD3 and anti-CD28 in the presence of high (50mM, n = 8) or normal (10mM, n = 8) glucose, analyzed at day 4. h, CD8+ T cells, two-sided unpaired t-test. i, CD4+ T cells, two-sided Mann-Whitney U-test. j-r, WT mice sublethally irradiated and transplanted with a Zbtb46-DTR bone marrow, followed, after 8 weeks, by administration of STZ or PBS. 2 weeks later, mice infected with 50pfu PR8 and treated, every other day, with PBS or DT: PBS+PBS (n = 10), PBS+DT (n = 10), STZ+PBS (n = 12) and STZ+DT (n = 8). j, Lung PR8 viral titer, Kruskall Wallis test and Dunn’s correction. k, Lung virus-specific NP34 Tetramer+CD8+ T cells, Kruskall Wallis test and Dunn’s correction. l, Lung CD4+ T cells, Kruskall Wallis test and Dunn’s correction. m, Lung CD8+ T cells, one-way ANOVA and Holm-Sidak correction. n, Lung B cells, Kruskall Wallis test and Dunn’s correction. o, Lung GC B cells, Kruskall Wallis test and Dunn’s correction. p, Lung cDC1, Kruskall Wallis test and Dunn’s correction. q, Lung cDC2, Kruskall Wallis test and Dunn’s correction. r, Lung CD64+ DC, Kruskall Wallis test and Dunn’s correction. s-x, WT and Akita mice intratracheally administered 100μg HDM, followed, 24 h later, by DC sorting and transfer to WT recipients (receiving WT DC (n = 9), Akita DC (n = 10), or no DC (n = 7)). Recipient mice intratracheally challenged by 10μg HDM daily from day 7-11, analyzed at day 14. s, Lung eosinophils and neutrophils, Kruskall Wallis test and Dunn’s correction. t, Lung T cells, one-way ANOVA and Holm-Sidak correction. u, Lung Ly-6Chigh and Ly-6Clowmonocytes, one-way ANOVA and Holm-Sidak correction. v, Lung cDC1, Kruskal Wallis test with Dunn’s correction. w, Lung cDC2, one-way ANOVA and Holm-Sidak correction and CD64+ DC, Kruskal Wallis test with Dunn’s correction. x, CD4+ T cell IL-5 and IL-13 expression, Kruskall Wallis test and Dunn’s correction, and CD4+ T cell IFNγ expression, one-way ANOVA and Holm-Sidak correction. All data mean+s.e.m. PAS, Periodic acid-schiff; DT, diphtheria toxin.

Extended Data Fig. 10 Lung DC from diabetic animals induce an attenuated T cell response.

a-h, WT and Akita mice intratracheally administered 100μg HDM, followed, 24h later, by DC sorting and transfer to WT recipients (receiving WT DC (n = 9), Akita DC (n = 10), or No DC (n = 7)). Recipient mice intratracheally challenged by 10μg HDM daily from day 7-11, analyzed at day 14. a, GATA3+CD4+ T cells, Kruskall Wallis test and Dunn’s correction. b, Ki-67+ T cells, Kruskall Wallis test and Dunn’s correction. c, Frequency of lung IL-10+CD4+ T cells, one-way ANOVA and Holm-Sidak correction, and IL-17A+CD4+ T cells, Kruskal Wallis test with Dunn’s correction. d, Frequency of lung RORγt+CD4+ T cells, T-bet+CD4+ T cells, Kruskal Wallis test with Dunn’s correction, and FoxP3+CD4+ T cells, one-way ANOVA and Holm-Sidak correction. e-g, WT and Akita mice intratracheally administered UV-inactivated PR8, followed 20 h later, by DC sorting and trasnfer to WT recipients (receiving WT DC (n = 6), WT+Akita DC (n = 6)). 10 days later, recipients infected with 500pfu PR8. e, Lung CD4+ T cells and CD8+ T cells, two-sided unpaired t-test. f, Frequency of T-bet+ cells, unpaired t-test. g, PR8 viral RNA, Mann Whitney test, WT+WT DC (n = 15), WT+Akita DC (n = 13), pooled from 2 experiments. h-o, WT lung DC incubated with high (50mM)/normal (10mM) glucose and 2-DG/vehicle for 20h, two-way ANOVA and Holm-Sidak correction. h, lung DC viability (n = 3/group). i-k, cDC1 (n = 4/group). i, CD40 MFI. j, CD80 MFI. k, CD86 MFI. l-n, cDC2 (n = 4/group). l, CD40 MFI. m, CD80 MFI. n, CD86 MFI. o, IL-12+ cells (n = 4/group). p-r, WT lung cDC1 incubated for 20 h with high (50mM, n = 12)/normal (10mM, n = 12) glucose, and high (50mM) glucose+2-DG (n = 3)/normal (10mM) glucose+2-DG (n = 3), then co-cultured with OT-I-CD8+ T cells for 4 days in normal (10mM) glucose (in the absence of 2-DG), two-way ANOVA and Holm-Sidak correction. p, CD8+ T cells. q, TNF+ cells. r, IFNγ+ cells. s-u, WT Lung cDC2 incubated for 20 h with high (50mM, n = 12)/ normal (10mM, n = 12) glucose, and high (50mM) glucose+2-DG (n = 3)/normal (10mM) glucose+2-DG (n = 3), then co-cultured with OT-I-CD8+ T cells for 4 days in normal (10mM) glucose (in the absence of 2-DG), two-way ANOVA and Holm-Sidak correction. s, CD8+ T cells. t, TNF+ cells. u, IFNγ+ cells. v, Mice intraperitoneally administered 250 mg/kg 2-DG (n = 5) or PBS (n = 5). Blood glucose measured at indicated time-points, two-sided unpaired t-test for each time-point. All data mean+s.e.m.

Extended Data Fig. 11 Diabetes does not impact lung DC lipid metabolism or glucose uptake.

a, Mice infected with 200pfu PR8 and intraperitoneally administered 250 mg/kg 2-DG (n = 10) or PBS (n = 10) daily. Survival, log-rank Mantel-Cox test. b-k, Mice infected with 50pfu PR8 and intraperitoneally administered 250 mg/kg 2-DG (n = 7) or PBS (n = 10) daily. b, Lung PR8 viral RNA, two-sided Mann-Whitney U-test. c, Lung virus-specific NP34 Tetramer+CD8+ T cells, two-sided unpaired t-test. d, Lung CD8+ T cells and CD4+ T cells, two-sided unpaired t-test. e, Frequency of IFNγ+CD8+ T cells, two-sided unpaired t-test. f, Lung Ki-67+CD8+ T cells, two-sided unpaired t-test. g, B cells,two-sided unpaired t-test. h, GC B cells, two-sided unpaired t-test. i, Lung cDC1, two-sided unpaired t-test. j, Lung cDC2, two-sided Mann-Whitney U-test and CD64+ DC, two-sided unpaired t-test. k, Lung Ki-67+cDC1, Ki-67+cDC2 and Ki-67+CD64+ DC, two-sided unpaired t-test. l-n, Mice intraperitoneally administered 250 mg/KG 2-DG (n = 10) or PBS (n = 10) daily for 10 days, two-sided unpaired t-test. l, Lung cDC1. m, Lung cDC2 and Lung CD64+ DC. n, Lung T cells. o-p, Seahorse ECAR analysis of lung DC, WT (n = 8) and Akita (n = 6) mice, two-sided unpaired t-test. q, Seahorse OCR of lung DC, WT (n = 12) and Akita (n = 8) mice, AUC, two-sided Mann-Whitney U-test. r-s, scRNA-seq of lung DC of genes involved in beta-oxidation. Heatmap shows z-scores of mean scaled and log normalized gene expression of cells in different DC. t, Lung DC incubated with 11 mM 13C-glucose. Intracellular pyruvate measured after 6 h (isotopologue fractional enrichment), 3 pooled experiments, two-sided paired t-test. All data mean+s.e.m. AUC, area under the curve; OCR, oxygen consumption rate.

Extended Data Fig. 12 Diabetes-induced hyperacetylation impairs lung DC.

a-c, WT (n = 5) and Akita (n = 5) mice intravenously administered 5 mM 2-NBDG. Frequency of 2-NBDG+ cells within immune cell compartments, two-sided unpaired t-test. d, WT lung cDC1 incubated with high (50mM) or normal (10mM) glucose and BMS303141 for 20h, then co-cultured for 4 days with OT-I-CD8+ T cells in normal (10mM) glucose (in the absence of BMS303141): High glucose+DMSO (n = 12), normal glucose+DMSO (n = 9), high glucose+BMS303141 (n = 16), normal glucose+BMS303141 (n = 9). Frequency of Ki-67+CD8+ T cells, two-way ANOVA and Holm-Sidak correction. e-f, WT lung cDC2 incubated with high (50mM) or normal (10mM) glucose and BMS303141 for 20h, then co-cultured for 4 days with OT-I CD8+ T cells in normal (10mM) glucose (in the absence of BMS303141): High glucose+DMSO (n = 8), normal glucose+DMSO (n = 7), high glucose+BMS303141 (n = 8), normal glucose+BMS303141 (n = 8), two-way ANOVA and Holm-Sidak correction. e, CD8+ T cells. f, Ki-67+CD8+ T cells. g-h, WT lung DC incubated with 10 mM DCA (n = 12) or vehicle (n = 12) for 20h, then co-cultured for 4 days with OT-I-CD8+ T cells in normal (10mM) glucose. g, CD8+ T cells, co-cultured with cDC2, two-sided Mann-Whitney U-test. h, IFNγ+CD8+ T cells, co-cultured with cDC1, two-sided unpaired t-test, IFNγ+CD8+ T cells, co-cultured with cDC2, two-sided Mann-Whitney U-test. i-k, OT-I-CD8+ T cells, co-cultured for 4 days with lung WT cDC1 (n = 15), WT cDC2 (n = 15), Pdk2-4−/− cDC1 (n = 11) or Pdk2-4−/− cDC2 (n = 12), two-sided unpaired t-test. i, CD8+ T cells, co-cultured with cDC2. j, IFNγ+CD8+T cells, and k, Ki-67+CD8+ T cells, cocultured with cDC1 and cDC2. l-m, CUT&Tag chromatin profiling of lung DC from naïve WT and Akita mice. Scatter plots with mean normalized reads in peaks in WT and Akita, red lines x = y. l, H3K27ac. m, H3K27me3. n, Flow cytometry of lung DC, representative histograms of H3K27ac. o-s, WT Lung DC incubated with high (50mM) or normal (10mM) glucose and 10mM ANA for 20h, then co-cultured for 4 days with OT-I CD8+ T cells in normal (10mM) glucose (in the absence of inhibitor), two-way ANOVA and Holm-Sidak correction. o-p, cDC1 incubated with high (50mM) glucose+DM (n = 6), normal (10mM) glucose+DM (n = 8), high (50mM) glucose+ANA (n = 8), normal (10mM) glucose+ANA (n = 8). o, Ki-67+CD8+ T cells. p, IFNγ+CD8+ T cells. q-s, cDC2 incubated with high (50mM) glucose+DM (n = 9), normal (10mM) glucose+DM (n = 8), high (50mM) glucose+ANA (n = 8), normal (10mM) glucose+ANA (n = 8). q, CD8+ T cells. r, Ki-67+CD8+ T cells. s, IFNγ+CD8+ T cells. t-w, WT and Akita mice infected with 50pfu PR8 and administered 5 mg/kg ANA or DMSO daily from 3 days before infection, analyzed at 10 d.p.i.: WT+DMSO (n = 12), WT+ANA (n = 12), Akita+DMSO (n = 13), Akita+ANA (n = 14), pooled data from 2 experiments. t, cDC1, Kruskal Wallis with Dunn’s correction. u, cDC2, Kruskal Wallis test with Dunn’s correction. v, CD64+ DC, one-way ANOVA with Holm-Sidak. w, Ki-67+cDC1, one-way ANOVA with Holm-Sidak. x-z, Mice intraperitoneally administered STZ or PBS, then infected with 50pfu PR8 and administered 5 mg/kg ANA/vehicle (DM) daily from 3 days before infection, analyzed at 10 d.p.i.: PBS+DM (n = 13), PBS+ANA (n = 10), STZ+DM (n = 10), STZ+ANA (n = 11), pooled data from 2 experiments. x, cDC1, Kruskal Wallis with Dunn’s correction. y, cDC2, one-way ANOVA with Holm-Sidak. z, CD64+ DC, one-way ANOVA with Holm-Sidak. All data mean+s.e.m. AM, Alveolar macrophage.

Supplementary information

Supplementary Information

Supplementary Figs. 1–16.

Reporting Summary

Supplementary Table 1

P values and statistics.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Nobs, S.P., Kolodziejczyk, A.A., Adler, L. et al. Lung dendritic-cell metabolism underlies susceptibility to viral infection in diabetes. Nature 624, 645–652 (2023). https://doi.org/10.1038/s41586-023-06803-0

- Received: 08 November 2022

- Accepted: 31 October 2023

- Published: 13 December 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06803-0

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

The lung environment controls alveolar macrophage metabolism and responsiveness in type 2 inflammation

Article 01 April 2019

Freya R. Svedberg, Sheila L. Brown, … Andrew S. MacDonald

Deficiency in the glycosyltransferase Gcnt1 increases susceptibility to tuberculosis through a mechanism involving neutrophils

Article Open access 13 March 2020

Kaori L. Fonseca, Ana Raquel Maceiras, … Margarida Saraiva

Impaired ketogenesis ties metabolism to T cell dysfunction in COVID-19

Article 28 July 2022

Fotios Karagiannis, Konrad Peukert, … Christoph Wilhelm

What makes people with diabetes more susceptible to serious lung infections?

Nature Research Briefing 13 Dec 2023

Article Open access Published: 22 November 2023

Structural insights into intron catalysis and dynamics during splicing

Ling Xu,

Tianshuo Liu,

Kevin Chung &

…

Anna Marie Pyle

Show authors

Nature volume 624, pages 682–688 (2023)

- 12k Accesses

- 69 Altmetric

- Metrics details

Abstract

The group II intron ribonucleoprotein is an archetypal splicing system with numerous mechanistic parallels to the spliceosome, including excision of lariat introns1,2. Despite the importance of branching in RNA metabolism, structural understanding of this process has remained elusive. Here we present a comprehensive analysis of three single-particle cryogenic electron microscopy structures captured along the splicing pathway. They reveal the network of molecular interactions that specifies the branchpoint adenosine and positions key functional groups to catalyse lariat formation and coordinate exon ligation. The structures also reveal conformational rearrangements of the branch helix and the mechanism of splice site exchange that facilitate the transition from branching to ligation. These findings shed light on the evolution of splicing and highlight the conservation of structural components, catalytic mechanism and dynamical strategies retained through time in premessenger RNA splicing machines.

Similar content being viewed by others

Main

Splicing lies at the heart of RNA metabolism in eukaryotes. During this indispensable stage of gene expression, introns are removed from premessenger RNA transcripts to generate mature messenger RNAs (mRNAs)1,3,4 (Fig. 1a). The modern spliceosome, the molecular machine that executes the splicing reaction, is thought to originate from the same ancestral molecule as the self-splicing group II introns that are still commonly found in bacteria and organelles of plants and fungi[2]. Group II introns are large ribozymes that catalyse their own excision from precursor RNA transcripts[5]. Both splicing machineries form a conserved active site that hosts the catalytically essential heteronuclear metal ion core6,7. Moreover, they both branch using a bulged adenosine nucleophile, forming the distinctive lariat intron featuring a 2′,5′-linked phosphodiester linkage. Intron D4 contains an open reading frame (ORF) that encodes a specialized multidomain protein (the ‘maturase’, Fig. 1b,c) which shares strong structural similarity to Prp8, a central protein component of the U5 snRNP8,9. Through formation of a ribonucleoprotein (RNP) holoenzyme with the parent intron RNA, the maturase facilitates intron splicing out of the transcript as well as retrohoming into new genomic loci[10].

Fig. 1: CryoEM reconstructions of a group IIC RNP undergoing the branching reaction.

a, Cartoon of group II RNP splicing. b, Domain organization of the intron RNA and its maturase. c, Secondary structure of the intron, with annotated tertiary interactions. d, A GelRed-stained 5% urea–polyacrylamide gel electrophoresis (PAGE) gel (top) and a SYPRO Ruby-stained SDS–PAGE gel (bottom) showing various conditions used to obtain samples for cryoEM. Lane 1 is the size marker for RNA (top) and protein (bottom). Lane 2 is the marker showing migration of the precursor RNA. Lanes 3 and 6 are the reaction ladders showing migration of the linear and lariat products, respectively. Lanes 4 and 5 are independent cryoEM samples captured at various reaction stages. e, Composite cryoEM maps of the prebranching (pre-1F), preligation (pre-2F) and postligation (post-2F) RNP complexes.

In light of these structural and mechanistic similarities, group II intron RNPs have become a prototypical system for studying the general biochemical principles of RNA splicing and the molecular evolution of splicing machines[11]. Despite advances in the visualization of group II intron RNAs7,12,13,14 and RNPs15,16,17, the structural organization of group II intron systems as they undergo branching and then coordinate the two steps of splicing has remained elusive. Therefore, it is unclear how group II introns properly recognize the branchpoint and the 5′-splice site (5′SS) and how maturases facilitate the branching reaction. These questions are of vital importance as they provide clues on the origin of intron branching, which is among the most fundamental reactions in RNA biology.

To visualize the conformational states along the group II intron RNP splicing pathway, we chose the group IIC intron from Eubacterium rectale and its encoded maturase, MarathonRT[18], as the model system. The maturase acts as a branching switch that shifts the intron splicing pathway from hydrolysis to branching (Extended Data Fig. 1a–d). Here, we used single-particle cryogenic electron microscopy (cryoEM) to obtain the structures of the RNP at each sequential stage during splicing. By capturing the RNP in the state immediately before branching (3.0 Å overall), we visualized how the branchpoint adenosine (bpA) and the splice site (SS) are held in place through molecular interactions between the branch helix and conserved regions of the RNP. Our structural observations reveal a close resemblance between group II intron RNPs and the spliceosome in terms of branchpoint recognition and branch helix positioning. We also gained unique insights into the strategy by which the attacking 2′-OH is precisely positioned in activation distance to the catalytic metal M1, ready for nucleophilic attack. This high-resolution view enables us to construct a complete and catalytically relevant molecular picture of the splicing active site before branching, which has largely eluded structural characterization despite earlier hints[19].

In addition to the prebranching RNP structure, we present the RNP structures preceding and following the exon ligation step. These structures allow us to visualize large movements of the branch helix, local movements of the branchpoint and the SS exchange that occurs between the two steps of splicing. The conformational dynamics of the spliceosome branch helix recapitulates that of the group II intron, thereby demonstrating that branch helix dynamics are a conserved physical attribute that is codified in splicing machines.

Capturing group II intron RNP in action

To elucidate the mechanism of forward splicing through branching, we sought to visualize structures of the group II RNP complex at each stage along the branching pathway. The two chemical steps of group II intron splicing are spontaneous and do not require energy input or step-specific factors. It is therefore challenging to stall the reaction without disrupting the active site and earlier attempts caused conformational distortions that made it difficult to discern the exact molecular mechanism of branching[20]. To resolve this, we incubated splicing precursor constructs in the presence of maturase protein, replacing Mg2+ with Ca2+ to yield complexes stalled in the precursor and branching intermediate states (Fig. 1d and Extended Data Fig. 1e,f). To obtain the postligation RNP, we assembled the lariat apoRNP[17] with an oligonucleotide equivalent to the ligated exon, thereby enabling us to investigate the structural changes upon completion of intron splicing (Fig. 1d).

The corresponding RNP samples were vitrified on grids and appeared as monodispersed particles on cryoEM micrographs, suitable for structure determination. As these samples show preferred orientation, we used the Chameleon system (Spotiton)21,22 and combined this with tilted datasets[23] to obtain a uniform angular distribution (Extended Data Figs. 2 and 3). The increased diversity of particle orientations allowed us to generate two distinct, isotropic maps. Upon inspection of the two reconstructions, we assigned the corresponding maps to the prebranching (pre-1F) and preligation (pre-2F) states, respectively (Fig. 1e). The resolution of these maps approaches 2.8 and 2.9 Å, respectively, for the catalytic core (Extended Data Fig. 3). We obtained a three-dimensional (3D) reconstruction for the postligation RNP (post-2F) and the resolution of the catalytic core was determined to 2.9 Å (Fig. 1e and Extended Data Fig. 4). This collection of structures allows us to present a full molecular picture of the group II intron RNP as it proceeds along the branching pathway.

Positioning of the branch helix

To splice through the branching pathway, the group II RNP forms intramolecular RNA and intermolecular RNA–protein interactions that precisely arrange the branch helix (D6) in the branching-competent conformation (Fig. 2a and Supplementary Video 1). The intron scaffold domain, D1, contributes to this by forming an extended, interlocked interaction network between D1c and D6 (denoted ν–ν′) (Fig. 2b). This network features a long-range base pair between G86 and C601, both of which are bulged nucleotides with strong conservation signatures (Fig. 2c and Extended Data Fig. 5a,b). Consistent with their significance in D6 positioning, deletion of G86, C601 or both, leads to branching defects, whereas substituting this GC pair for an AU pair partially rescues branching (Fig. 2d). A wobble pair between G84 and U104 in D1c anchors another intricate molecular network around C601 to further restrain the conformational sampling of D6. In agreement with our structural observations, a G84A/G86A dual mutation, which does not alter D1c secondary structure, has the most pronounced deleterious effect (Fig. 2d). Hence, our results highlight the active role of interdomain RNA interactions in proper positioning of D6.

Fig. 2: Interactions of D6 with the intron RNP before lariat formation.

a, Interaction network surrounding the D6 helix before branching (boxed elements are labelled with the figure panel designations described next). b, Newly discovered long-range RNA interaction (ν–ν′) that positions D6. c, Conservation of nucleotides involved in b. d, A denaturing radioanalytical splicing gel showing effects of intron mutants in the presence of WT maturase protein. Individual data points representative of n = 4 in vitro splicing assays are shown. e, Intron–maturase interaction cluster at the basal region of the D6 helix. f, Intron–maturase interaction cluster at the central region of the D6 helix. g, Intron–maturase interaction cluster at the distal region of the D6 helix. h, A denaturing radioanalytical splicing gel demonstrating effects of maturase mutants on promoting branching of WT intron construct. Individual data points representative of n = 4 in vitro splicing assays are shown.

On the opposite side of the D6 helix, the thumb and DBD domains of the maturase protein compose another extensive RNA–protein intermolecular interface, where we visualized three clusters of interactions. The first cluster (Trp310, Ser313 and Gln359) is located at the basal stem of D6 (Fig. 2e) and its disruption, through mutation to alanine, abolishes intron branching (Fig. 2f). At the junction loop between the thumb and DBD domain a second cluster of residues (Thr362 and Asn365) grasps the central section of D6 adjacent to the branch site and the ribozyme active site. In addition to phosphate backbone interactions, a highly conserved lysine residue (Lys361) inserts into the main groove of D6 and makes direct contact with the 5′SS (G1:N7), juxtaposing the first-step nucleophile with the scissile phosphate (Fig. 2e and Extended Data Fig. 5c). Consistent with this structural observation, a Lys361 single alanine mutant is sufficient to eliminate branching activity, as does the Lys361/Thr362/Asn365 triple mutant (Fig. 2f). Finally, the side chains of Lys372 and Arg377 in the DBD domain grip the distal, upper stem of D6 and mutations introduced at these sites compromise branching. Using this vast molecular network, the maturase protein stabilizes the branching-competent conformation of D6 and brings the 5′SS adjacent to the branchpoint, thereby explaining why it is indispensable for promoting branching (Extended Data Fig. 1).

In the D6-docked state, one observes an interplay between the intron 5′SS and the branch helix, where the first two nucleotides expose their Watson–Crick edges to engage in tertiary interactions with D6. Specifically, G1:O6 engages the 2′-OH of C633, the nucleotide next to the branchpoint, which secures D6 and brings the branchpoint close to the 5′SS. U2 further strengthens contacts between the 5′SS and D6 through a base triple interaction with the G599-C629 base pair (Extended Data Fig. 6a). The pre-1F structure therefore provides a glimpse into the group II intron 5′SS and establishes its role in branch helix positioning, explaining the conservation signature of the group II intron 5′SS (ref. [24]).

Branchpoint recognition and dynamics

Having elucidated the mechanism by which the branch helix is docked in the prebranching conformation, we next sought to unveil the branchpoint recognition strategy. This question is of vital importance, as stringent branchpoint use is a hallmark of group II intron splicing[25] and yet the interaction network that recognizes and activates the branchpoint nucleotide for catalysis has eluded structural characterization.

Our prebranching map, with a local resolution of 2.8 Å around the branchpoint, allows confident model building which reveals the structural basis of branch site recognition. We show that the bpA (A632) is recognized by means of a base triple interaction with the G598-C630 base pair. The exocyclic amine of the bpA (bpA:N6) forms a crucial hydrogen bond with O2 of C630 (Fig. 3a), consistent with previous chemical genetics studies[25]. The interaction partner (C630) is located two nucleotides upstream and, based on covariation analysis of group II introns[26], it is almost exclusively a pyrimidine. The 2′-OH of C630 forms an extra hydrogen bond with bpA:N1. This interaction serves as an extra molecular lock to hold the branch site in place. Intriguingly, the molecular recognition pattern we observe in group II introns is identical to that reported in structures of the spliceosome6,27. Recent models for the yeast C complex[6] (postbranching) revealed the same molecular interaction between the bpA and a highly conserved uridine located two nucleotides upstream (Fig. 3b), which can reasonably fit in the density of the yeast B* complex (prebranching). A similar bpA recognition strategy has also been proposed in the C complex of the human spliceosome[27]. Our structure therefore unveils a mechanistic parallel in splicing machines for defining the branchpoint, which seems to be hard-coded by molecular evolution.

Fig. 3: Molecular recognition of the branchpoint A and D6 dynamics.

a, Interactions that specify the bpA before branching. b, Positioning of the bpA in the yeast spliceosome C complex. c, Interaction network surrounding the bpA postligation. d, Aligned D6 helices showing conformational movement of the bpA during stages of splicing. e, Conformational rearrangement of D6 from branching to exon ligation. f–h, Secondary structure schematic with annotated tertiary contacts of the RNP in the pre-1F (f), the pre-2F (g) and the post-2F state (h). Yellow cartoon batons indicate the position of the maturase DBD helices.

Next, we sought to visualize the local conformational dynamics of the bpA along the branching pathway. In the pre-1F state, the bpA adopts an unusual conformation that causes it to point toward the main groove of the branch helix, through a base triple interaction. This conformation leads to significant distortion of the bpA sugar–phosphate backbone (Fig. 3a,d), which places the 2′-OH next to the scissile phosphate. After branching, as the complex enters the preligation state, the bpA flips to the opposite side of the branch helix and points toward the 3′-end of D6 (Fig. 3d and Supplementary Video 2). This dramatic conformational rearrangement of the bpA relaxes the backbone distortion, potentially releasing free energy that compensates for the energetic cost of disengaging interactions that originally anchored the bpA[5]. Upon exon ligation, the 3′-end of the intron (C635 and G636) moves further towards the branch site and makes direct contact with the Hoogsteen face of the bpA (Fig. 3c). These extra molecular interactions, formed after exon ligation, limit the conformational flexibility of the bpA and mark the termination of the splicing pathway.

Branch helix conformational dynamics

In addition to the local dynamics of the bpA, comparison of the intron RNP structures reveals a set of tertiary contact rearrangements needed to coordinate the two sequential steps of splicing (Fig. 3e–h and Supplementary Video 3). In the pre-1F state, the intron recognizes the 5′-exon through the EBS1–IBS1 interaction and the branch helix adopts the D1c- and maturase-docked conformation (Fig. 2a), which we refer to as the ‘up’ position hereafter (Fig. 3e). In this arrangement, an array of long-range interactions form between J4/5 (A559 and A560) and J5/6 (U591, G592, U593) which participate in a coordinated series of interactions (Extended Data Fig. 6c). This network begins with a canonical base pair (A560-U591) and continues with a non-canonical pairing, in which the Hoogsteen edge of A559 interacts with the sugar edge of G592 and is capped by the final nucleotide of J5/6, U593, which stacks beneath A559. Owing to the interactions that pull D6 into the up position and the constraints imposed by the J5/6 interaction network, the phosphate backbone connecting D5 and D6 adopts a bent conformation, flipping adjacent nucleotides to opposing sides (Extended Data Fig. 6d).

After branching, D6 undergoes a substantial structural rearrangement that involves an approximately 90° swing, to the ‘down’ position (Fig. 3e). Through this process, the intron pulls the 5′SS and the newly formed lariat bond about 21 Å out of the active site and exchanges it for the 3′SS, thereby preparing the active site for exon ligation. During this transition, the ν–ν′ tertiary interaction and D6–maturase contacts are disrupted. In the resulting pre-2F structure, D6 docks onto D2, engaging π–π′, which latches onto the branch helix, thereby pulling D6 and the covalently linked 3′-exon into position (Fig. 3g). This allows formation of the EBS3–IBS3 base pairing that defines the 3′SS (U(+1)-A231) (Fig. 3f,g). Comparison of the catalytic D5 helix in the pre-1F and pre-2F structures reveals that it remains stationary within the D1 scaffold (root mean square deviation of 0.5 Å). Instead, movement of the D6 helix hinges on the J5/6 linker and appears as motion of the branch helix relative to a fixed RNP body. Swinging of D6 into the plane of the RNP relaxes the bent conformation of J5/6 (Extended Data Fig. 6e), enabling an exchange of substrates in the active site and driving the branching reaction forward[5]. The structural importance of J5/6 in branching is consistent with mutational studies that investigated its biochemical function in positioning of the branch helix28,29. Further movement of D6 is observed on completion of splicing, where there is minor motion of the D6 3′-end, which tucks inwards, allowing engagement of γ–γ′ (A327-U638) (Fig. 3h). Hence, our structures provide detailed molecular insights into the conformational rearrangements and sequential transitions that are required for branching and SS specification during group II intron splicing.

Catalytic mechanism of intron splicing

Having revealed the dynamical strategies used by the group II intron RNP throughout the branching pathway, we next sought to visualize the chemical catalytic mechanisms for each step. Catalysis of the branching reaction is potentiated by a heteronuclear metal ion core organized around the catalytic triplex and the two-nucleotide bulge of D5 (Fig. 4a,b). Through precise positioning of D6 and formation of intra-D6 interactions (Fig. 2a and Fig. 3f), the first-step nucleophile (2′-OH of the bpA) is precisely positioned by catalytic metal M1 and placed in the activation distance (2.3 Å), where it is poised for the nucleophilic attack (Fig. 4b). Remarkably, the attacking 2′-OH nucleophile in the pre-1F structure occupies an identical position to that of the water nucleophile in an earlier pre-hydrolytic structure (Fig. 5a), which provides an unambiguous explanation for the competitive nature of the two splicing pathways[30]. The scissile phosphate of the 5′SS, between U(−1) and G1, adopts the same sharply kinked conformation previously observed for the hydrolytic, precatalytic state[7]. The pro-Rp oxygen of the scissile phosphate coordinates both M1 and M2 whereas the 3′-bridging oxygen is in direct contact with M2, facilitating departure of the 3′-oxyanion leaving group. This high-resolution view of the active site in the prebranching state hence provides direct visualization of the two-metal ion mechanism for group II intron branching proposed three decades ago[31]. In addition, we identified two strong, globular densities around the divalent metal core, whose positions correspond to the previously identified monovalent ions, K1 and K2 (ref. [7]) (Extended Data Fig. 7a,b). Our findings therefore highlight the formation of a heteronuclear metal ion core as a general catalytic strategy fundamental to RNA splicing6,7,32.

Fig. 4: Molecular mechanism of group II RNP branching and exon ligation.

a, Organization of catalytic elements before branching. The bpA is juxtaposed to the 5′SS and poised for lariat formation. b, Zoomed in view of (a). c, Active site configuration before exon ligation. The 5′SS is primed for attack to ligate the exons. d, Zoomed in view of (c). e, Positioning of active site elements immediately after exon ligation. f, Zoomed in view of (e). Divalent metal ions are shown as green spheres.

Fig. 5: Mechanistic comparison of group II introns and the spliceosome.

a, Comparison of the overall fold of group IIC introns (top) and the aligned active sites of the Oceanobacillus iheyensis intron before hydrolysis (nucleophilic water in light blue) and the E. rectale intron before branching (bpA in dark red) (bottom). b,c, Conserved RNP interface and branch helix dynamics in group II RNPs (b) and spliceosomes (c) during the first- to second-step transition.

Upon cleavage of the 5′SS, D6 movement brings the first-step nucleophile and the now covalently linked G1 out of the active site (Fig. 3e and Fig. 4c,d). The first-step leaving group, the U(−1):3′-OH, remains tightly coordinated with catalytic metal M2 and becomes the activated second-step nucleophile. The second-step scissile phosphate between U638 and U(+1) then becomes visible in the pre-2F state, adopting the same precleavage kinked configuration (Fig. 4c,d). These data establish that the same active site is used for both splicing steps without modifying the catalytic ion configuration nor the metal-binding platform (Fig. 4b,d and Extended Data Fig. 7). Moreover, our structure of the post-2F state with the ligated exon bound (Fig. 4e,f) shows that the metal catalytic core remains well organized, whereby the 3′-bridging oxygen of U(−1) remains associated with M2 and the 3′-OH of U638 is coordinated with M1 (Fig. 4f).

Discussion

Splicing at the RNP interface

As revealed by our study, group II intron and spliceosome not only share structural and chemical components (Extended Data Fig. 7c) but also a conserved dynamical strategy for sequential rearrangement between the steps of splicing. Direct parallels can be drawn between the motions of the D6 helix in the group II intron and the branch helix in the spliceosome. Comparison of their identical branching states reveals the same 90° swing of the U2-intron branch helix during the transition from the branching B* complex to the exon ligation C* complex (Fig. 5b,c). Analogous conformational dynamics are observed in the group II intron holoenzyme, as it swaps SSs without disrupting the catalytic core, when transitioning between the steps of splicing. Remarkably, the branch helix swinging motion has equivalent centres of rotation to that of the spliceosome, whereby the J5/6 linker in the group II intron acts as a hinge, much like the corresponding U2/U6 linker in the spliceosome[29]. Intriguingly, as in the spliceosome (U2/U6), there are no conformational rearrangements of the catalytic triplex (Fig. 5b,c), which remains static through the stages of branching[19]. We now have direct evidence of a conserved dynamical mechanism of SS exchange by group II introns that has direct parallels with the spliceosome, strengthening the argument that group II introns and spliceosome share the same ancestry.

Despite the many features in common with group II introns, we identified a functional difference that provides an extra layer of regulation for the spliceosome. The first short α-helix located in the maturase DBD domain has a positively charged surface (Extended Data Fig. 8a) that is indispensable for spontaneous group II intron RNP branching (Fig. 2e,f). In contrast, whereas the equivalent helix in the linker domain of Prp8 adopts a highly similar pose (Fig. 5b,c), the contact surface is negative to neutral in charge (Extended Data Fig. 8b). This marked difference between the maturase and Prp8 has evolutionary implications. On one hand, the maturase is the lone protein cofactor necessary for proper positioning of the D6 branch helix. However, the spliceosome requires recruitment of step 1 specific factors, such as Yju2, to activate branching[19]. We can now structurally rationalize the need for Yju2 by comparing the maturase and Prp8 surfaces. The N terminus of Yju2 may compensate for the positive charges that are lost during molecular evolution from the maturase to Prp8 by forming a highly positively charged contact surface that interacts with the branch helix (Extended Data Fig. 8b). Intriguingly, the maturase side chain, Lys361, shown to be essential for intron branching in our study (Fig. 2f) has no equivalent in Prp8; whereas a highly conserved residue (Arg3 in Saccharomyces cerevisae and Homo sapiens)[33] at the N terminus of Yju2 plays a similar role in contacting G1 of the 5′SS. We therefore observe hints of a molecular evolutionary strategy that fragmented the single-protein RNP into a multiprotein splicing machine, which allows for fine tuning of RNA splicing as a regulated biological process.

Conservation of molecular recognition

The prebranching RNP structure presented in this study reveals the 5′SS and branchpoint recognition strategy used by group II introns, thereby providing critical insights into how splicing machinery maintains precise SS and branchpoint definition during molecular evolution.

We present the pre-attack conformation of the bpA in group II introns (Fig. 4a). This high-resolution view unambiguously explains the branch site recognition strategy used by group II introns. Instead of canonical base pairing, the intron resorts to a base triple (cis Watson–Crick/sugar edge interaction) formed between the bpA and a CG base pair located two nucleotides upstream (Fig. 3a). The interaction also serves to hold the bpA inwards, toward the major groove of the D6 branch helix, thereby limiting its conformational flexibility and correctly positioning its 2′-OH relative to the catalytic metal for activation. The same molecular recognition strategy is used by the spliceosome to anchor its bpA (Fig. 3b). This striking similarity provides molecular evidence that there is minimal change to the strategy for branchpoint definition during evolution from group II introns to the spliceosome.

Also, we revealed the molecular basis of group II intron 5′SS recognition. Through base–sugar interactions originating from G1 and a base triple interaction from U2 (Extended Data Fig. 6a), the intron 5′SS interlocks with the branch helix and closely contacts the branchpoint, preparing the system for branching. The abundance of molecular interactions surrounding the 5′SS also enforces stringent nucleotide identity requirements. Given the mechanistic parallel with the spliceosome (Extended Data Fig. 6b), we can now justify why the same 5′-GU motif[24] has persisted through time, highlighting that the strategy to define the 5′SS is so robust that it has withstood the forces of molecular evolution.

Group II RNP life cycle

By combining the cryoEM structures obtained in this study with previous mechanistic and structural work done on group II introns7,12,17,34, we can now propose a mechanism for the group II intron splicing life cycle (Supplementary Video 4), including excision from the flanking exons and retrohoming into DNA sites (Extended Data Fig. 9 and Supplementary Video 5). After translation of the maturase from the ORF, the protein facilitates RNA folding by binding to the D4a arm and interacting with D1, which folds first and acts as a scaffold35,36 for assembly of downstream domains. The D6 branch helix docks onto D1 through the intramolecular ν–ν′ interaction and engages the thumb/DBD domains of the maturase to stabilize the helix in the up position. Specific molecular interactions distinguish and lock the bpA and 5′SS into place, juxtaposing the 2′-OH against the scissile phosphate for nucleophilic attack through the heteronuclear metal ion active site. During the first stage of branching, the 2′,5′-phosphodiester bond is formed, exposing the 3′-OH of the 5′-exon for ligation. To exchange substrates, the branch helix then disengages ν–ν′ and the maturase–D6 interactions, permitting the bpA to pivot around its phosphate and the D6 helix to swing downwards (Supplementary Videos 2 and 3), where it forms the π–π′ tertiary interaction with D2. This pulls the lariat out of the active site and replaces it with the 3′SS, demarcated by the EBS3–IBS3 base pairing, thereby positioning the 3′-exon for ligation. Using the same active site, the 3′-OH is activated for nucleophilic attack, resulting in splicing of the exons. Following exon ligation, the D6 3′-tail tucks inward and the terminal nucleotide is secured by the γ–γ′ interaction. The ligated exons are then released and the liberated apoRNP retains its overall architecture, enabling it to remain primed for binding DNA substrates on the basis of shape and sequence complementarity for engagement in reverse splicing[17].

To undergo retrotransposition, we postulate that equivalent, conserved D6 and branch site motions are used[16] to achieve intron integration and substrate exchange using a persistent heteronuclear metal ion core. Given the proximity of the maturase reverse transcriptase active site to the 3′-end of the integrated intron, a logical hypothesis is that the 3′-end of the fully reverse spliced product is threaded into the maturase reverse transcriptase domain. Here, the protein, using an exogenous primer, begins target primed reverse transcription, unravelling the base pairing, disassembling the elaborate intron tertiary structure[37] and generating a complementary DNA strand to effectively copy and paste the RNA sequence into a new genomic site, thereby completing the intron life cycle. Further biochemical and structural work will be needed to evaluate this hypothesis and address the remaining mechanistic aspects of the group II RNP life cycle after branching.

Methods

Protein purification

Wild-type (WT) maturase protein (MarathonRT) was purified as previously described[17]. Briefly, the recombinant protein was cloned into a pET-SUMO vector which has an N-terminal 6×His-SUMO tag. The plasmid was transformed into Rosetta 2 (DE3) cells (MilliporeSigma), which were grown at 37 °C in LB medium supplemented with kanamycin and chloramphenicol. Cells were grown to an optical density of 1.5 in large 2 l cultures before induction with IPTG and shaking at 16 °C overnight. Cells were collected by centrifugation and resuspended in a lysis buffer (25 mM Na-HEPES, pH 7.5, 1 M NaCl, 10% glycerol and 2 mM β-mercaptoethanol (βME)) with dissolved protease inhibitor. Cells were lysed using a microfluidizer and the cell lysate was clarified to remove precipitants. The lysate was loaded onto an Ni-NTA column and was washed with lysis buffer and wash buffer (25 mM Na-HEPES, pH 7.5, 150 mM NaCl, 10% glycerol, 2 mM βME and 25 mM imidazole) before elution (25 mM Na-HEPES, pH 7.5, 150 mM NaCl, 10% glycerol, 2 mM βME and 300 mM imidazole). The SUMO tag was removed using ULP1 SUMO protease by incubating at 4 °C for 1 h. After tag cleavage, the protein was loaded onto a HiTrap SP HP cation exchange column (Cytiva) equilibrated with buffer A (25 mM K-HEPES, pH 7.5, 150 mM KCl, 10% glycerol and 1 mM dithiothreitol). The protein was eluted by running a linear gradient to buffer B (25 mM K-HEPES, pH 7.5, 2 M KCl, 10% glycerol and 1 mM dithiothreitol). The peak fractions were pooled, concentrated to 5 ml and injected onto a HiLoad 16/600 Superdex 200 pg size exclusion column (Cytiva) and eluted using a SEC buffer (50 mM NH_4-HEPES, 150 mM NH_4Cl, 5 mM dithiothreitol and 10% glycerol). Peak fractions from the S200 column were pooled, concentrated to 5 mg ml−1, flash frozen under liquid nitrogen and stored at −80 °C. Mutant proteins (Trp310A/Gln359Ala, Lys361Ala, Lys361Ala/Thr362Ala/Asn365Ala and Lys372Ala/Arg377Ala) were also prepared in the same manner. The folding integrity of the WT maturase protein and all mutants were verified with an orthogonal activity assay involving the inherent reverse transcriptase capability[37] of these maturase proteins. All proteins were fully active for reverse transcription on long RNA templates.

RNA transcription and purification

The DNA sequence containing the T7 promoter, 100 nucleotides (nt) of 5′ exon, the intron (with ORF deletion) and 100 nt of 3′ exon followed by the BamHI cleavage site were cloned into the pBlueScript vector (Invitrogen) to give the pLTS01 plasmid, which is used for structural studies. For splicing assays, a longer 5′ exon (150 nt) construct (which was cloned into plasmid pCZ26) was used. The plasmids were linearized using BamHI (New England Biolabs) to generate the DNA template. Mutants of the construct were prepared using a PfuUltra II Hotstart PCR Master Mix (Agilent) mixed with 20 ng of plasmid, 10 pmole of forward and reverse primers with routine site-directed mutagenesis protocol. The sequences of the resulting plasmids were verified with Sanger sequencing (Quintara Biosciences).

RNA used for cryoEM sample preparation was prepared as previously described[38]. Briefly, in vitro transcription of the intron precursor RNA was performed using inhouse-prepared T7 RNA polymerase (P266L mutant) in a transcription buffer containing 40 mM Tris-HCl pH 8.0, 10 mM NaCl, 23 mM MgCl_2, 2 mM spermidine, 0.01% Triton X-100, 10 mM dithiothreitol and 5 mM each rNTP. A total of 40 μg of linearized pLTS01 plasmid was added for each 1 ml of transcription and the reaction was incubated at 37 °C for 6 h before it was ethanol precipitated. The pellet was resuspended in water and mixed with an equal volume of 2× urea loading dye containing bromophenol blue and xylene cyanol, which was then loaded onto a 5% urea denaturing polyacrylamide gel to purify the intron precursor RNA. The band was visualized with ultraviolet shadowing, cut with a sterile blade, crushed with a sterile syringe and eluted in a gel elution buffer overnight (10 mM Na-MOPS pH 6.0, 300 mM NaCl and 1 mM EDTA). The eluted RNA was then ethanol precipitated, resuspended in an RNA storage buffer (6 mM Na-MES pH 6.0) to a final concentration of 50 μM and frozen at −80 °C for preparation of RNP samples.

The radiolabelled intron precursor RNA for the splicing assay was prepared using the previously described body-labelling protocol[39]. Briefly, the intron precursor RNA transcripts were body-labelled using inhouse-prepared T7 RNA polymerase and 50 μCi of [α-32P]-UTP (PerkinElmer) in a transcription buffer containing 40 mM Tris-HCl pH 8.0, 10 mM NaCl, 15 mM MgCl_2, 2 mM spermidine, 0.01% Triton X-100, 10 mM dithiothreitol and 3.6 mM each rNTP (except UTP, which was at 1 mM) and 5 μg of linearized pCZ26 plasmid. Mutant intron precursor RNAs (ΔG86, ΔC601, ΔG86/ΔC601, G86A/C601U and G84A/G86A) were prepared in the same manner. The reaction was incubated at 37 °C for 1.5 h and followed by purification on a 5% urea denaturing polyacrylamide gel. The band corresponding to precursor RNA was visualized through brief exposure of a phosphor storage screen and the screen was subsequently imaged using an Amersham Typhoon RGB imager (Cytiva). The band was cut and eluted overnight in the gel elution buffer. The radiolabelled RNA was then ethanol precipitated, resuspended in the RNA storage buffer to a final concentration of 100 nM and stored in −20 °C for splicing assays.

In vitro forward splicing assay

Radiolabelled intron precursor RNA (and mutants) were mixed with purified maturase protein (and mutants) under near-physiological condition (50 mM K-HEPES pH 7.5, 150 mM KCl and 5 mM MgCl_2) to perform the in vitro forward splicing assay. To do so, radiolabelled intron RNA was first mixed with buffer and water and heated to 95 °C for 1 min, after which it was returned to 37 °C for 5 min. Potassium chloride and maturase protein stocks were added and the sample was incubated at 37 °C for another 5 min. Magnesium chloride stock was subsequently added to the mixture to initiate the splicing reaction. The final concentration of radiolabelled intron precursor RNA was 5 nM and maturase protein was 20 nM. After incubation at 37 °C for 1 h, 2 μl of the reaction mixture was taken out and quenched by mixing with an equal volume of 2× formamide loading dye (72% (v/v) formamide, 10% sucrose, 0.2% bromophenol blue dye, 0.2% xylene cyanol dye and 50 mM EDTA) precooled on ice. Samples were analysed on a 5% urea denaturing polyacrylamide gel. The gel was dried and used to expose the phosphor storage screen overnight. The screen was then imaged on an Amersham Typhoon RGB imager (Cytiva). The bands were quantified using ImageQuant TL 8.2 (Cytiva).

Group II intron RNP sample preparation

To obtain the E. rectale intron–maturase RNP complex stalled in the pre-1F and pre-2F states, 0.9 ml of reaction with 5 μM purified intron precursor RNA, 10 μM purified maturase protein was conducted in a buffer containing 50 mM NH_4-HEPES pH 7.5, 150 mM NH_4Cl, 10 mM CaCl_2 and 5 mM dithiothreitol. To do so, purified intron precursor RNA stock was mixed with buffer and water and heated to 95 °C for 3 min. It was then incubated at 37 °C for 5 min. The ammonium chloride and calcium chloride stocks were then added to refold the RNA at 37 °C for 10 min. After that, the dithiothreitol and maturase stocks were added to the reaction mixture and the reaction was incubated at 37 °C for 1 h with shaking at 300 rpm on a Thermomixer, after which it was centrifuged at 10,000g for 2 min to pellet the precipitates. The supernatant was subsequently loaded onto a Superdex 200 Increase 10/300 GL column (Cytiva) pre-equilibrated with the buffer containing 50 mM NH_4-HEPES pH 7.5, 150 mM NH_4Cl, 10 mM CaCl_2 and 5 mM dithiothreitol. The elution peak was pooled together and concentrated to about 4 mg ml−1 using an Amicon concentrator (10 kDa MWCO) (MilliporeSigma). The concentrated sample was used to prepare the cryo grids (see below).

To obtain the E. rectale intron–maturase RNP complex in the post-2F state, the lariat intron apoRNP was first purified as previously described[17] and the peak fraction was collected. A 1.2× molar excess of a synthetic RNA oligonucleotide with the sequence 5′-AUUUCUUUUGAAU-3′ (Integrated DNA Technologies) was then added to a final concentration of 600 nM, resulting in a final concentration of 500 nM for the RNP. The sample was incubated on ice for 10 min to allow formation of the ternary RNP complex, which was used to prepare cryo grids (see below).

All samples for cryoEM, precursor RNA and reaction ladders generated from self-splicing and maturase-mediate splicing were loaded onto a 5% urea denaturing polyacrylamide gel run in a Mini-PROTEAN tetravertical electrophoresis cell (Bio-rad) at 180 W for 50 min before staining with GelRed (Biotium) and imaging with the Cy3 channel on an Amersham Typhoon RGB imager (Cytiva).

Grid preparation and data collection

For cryoEM analysis of the E. rectale intron–maturase RNP complex stalled in the pre-1F and pre-2F states, 7 μl of purified sample was loaded onto the Chameleon system (SPT Labtech). A total of 40 nl of the sample solution was dispensed to the glow-discharged Quantifoil Active Cu 300-mesh R1.2/2 grids with Cu nanowires (SPT Labtech) and the grids were plunged into liquid ethane and frozen in liquid nitrogen about 400 ms after. The grids were screened on a Talos Glacios microscope (ThermoFisher) operating at 200 keV. Grids with sufficient collectable squares and minimal crystalline ice contamination were selected for data collection on a Titan Krios microscope (ThermoFisher) operating at 300 keV with a K3 summit direct electron detector (Gatan) and the data were obtained in the counting mode. SerialEM v.3.9 was used for data collection. Two datasets of 4,612 micrographs (at 30° tilt angle) and 7,777 micrographs (at 0° tilt angle) were collected for the pre-1F/pre-2F sample. A nominal magnification of 81,000× and a defocus range of −0.8 to −2.5 μm was used, giving an effective pixel size of 0.844 Å at the specimen level. Each micrograph was dose-fractionated to 40 frames with a total exposure time of 3.482 s and a frame exposure time of 0.0865 s, resulting in a total dose of 60 e−/Å^2.

For cryoEM analysis of the E. rectale intron–maturase RNP complex in the post-2F state, two separate approaches were taken. The first strategy used the Chameleon for grid preparation. Here, 8 μl of purified sample was loaded onto the Chameleon system (SPT Labtech). A total of 40 nl of the sample solution was dispensed to the glow-discharged Quantifoil Active Cu 300-mesh R1.2/2 grids with Cu nanowires (SPT Labtech) and the grids were plunged into liquid ethane and frozen in liquid nitrogen about 300 ms after. In parallel, grids were prepared using the Vitrobot. Here, 4 μl of the purified RNP sample was loaded onto plasma cleaned QuantiFoil Cu R1.2/1.3 300-mesh grids prepped inhouse with an extra layer of carbon. The grids were blotted and plunged into liquid ethane and frozen in liquid nitrogen using a condition of 100% humidity at 22 °C. SerialEM v.4.0 was used for data and micrographs were recorded on a Titan Krios microscope (ThermoFisher) operating at 300 kV equipped with a K3 Summit direct electron detector (Gatan) operating in counting mode. Two datasets of 13,740 micrographs (Chameleon) and 3,852 micrographs (Vitrobot) were collected for the post-2F sample. For the Chameleon dataset, a nominal magnification of ×105,000 and a defocus range of −1.0 to −2.5 μm was used, giving an effective pixel size of 0.832 Å at the specimen level. Each micrograph was dose-fractionated to 50 frames with a total exposure time of 1.5 s and a frame exposure time of 0.03 s, resulting in a total dose of 50.5 e−/Å^2. For the Vitrobot dataset, a nominal magnification of ×105,000 and a defocus range of −1.0 to −2.0 μm was used, giving an effective pixel size of 0.832 Å at the specimen level. Each micrograph was dose-fractionated to 48 frames with a total exposure time of 1.92 s and a frame exposure time of 0.04 s, resulting in a total dose of 50 e−/Å^2.

CryoEM data processing

Recorded video frames were processed using cryoSPARC v.3.4 (refs. 40,41). Motion correction and contrast transfer function (CTF) estimations were performed using default parameters in cryoSPARC. Exposures were curated and micrographs with obvious ice contamination, large motions or damaged areas were removed.

For the pre-1F and pre-2F samples, particle picking was done with the automated blob picker and filtered using consecutive rounds of two-dimensional (2D) classification. This subset of particles was used for Topaz (Topaz 0.2.4) training and picking on the combined 11,783 micrographs. Topaz picking yielded 1,289,915 particles which were subject to three rounds of 2D classification, leaving 847,534 particles which were extracted with a box size of 384 × 384 pixels. A total of 100,000 particles were selected for initial model creation, generating three reconstructions, of which one was selected as the reference for further classification. In the first round of 3D classification, eight classes were separated out, from which two groups of 422,728 particles and 234,625 particles corresponding to the pre-1F and pre-2F states were identified. Each group was subjected to another round of 3D classification, leaving a final subset of particles of 281,619 particles (pre-1F) and 234,625 particles (pre-2F, all particles were selected). Each group of particles was separately refined, yielding reconstructions of 3.0 and 3.1 Å, respectively. For each reconstruction, separate masked local refinements and local and global CTF refinement were conducted on the left and right halves to improve resolution of each. Four different maps were obtained for the left and right halves of the pre-1F and pre-2F reconstructions which have resolutions of 2.9, 3.1, 3.0 and 3.3 Å, respectively, as evaluated by the GSFSC with a cutoff of 0.143 (Extended Data Figs. 2 and 3).

A similar strategy was used to obtain the 3D reconstruction of the post-2F state. Briefly, particles were picked with an automated blob picker from the first dataset of 13,740 micrographs (Chameleon grid) and the results were filtered through several rounds of 2D classification and used as an input for Topaz training. This yielded 930,320 particles which were further cleaned through iterative rounds of 2D classification leaving 695,515 particles. Three initial models were generated with a subset of 100,000 particles out of which a single refinement with the best overall density was chosen. The 3D classification was used to separate the particles into ten classes from which a single class with 217,454 particles was selected. These particles were used to train a Topaz model and after iterative rounds of 2D classification cleaning, 361,978 particles remained. These particles were combined with 342,031 particles that were Topaz picked and filtered from a separate dataset of 3,852 micrographs (Vitrobot grid). The combined 722,394 particles were 2D classified and manually separated into groups corresponding to the dominant and less-represented views. The 70,000 randomized particles were taken from the dominant view group and aggregated with all particles from the ‘other’ views. Particles were extracted with a box size of 384 × 384 pixels. After an initial 3D refinement, 3D classification into ten classes yielded two classes with good overall density, which were used to generate an overall reconstruction of 3.0 Å. Similar to the other branching structures, masked local refinement was done on the separate halves along with local and global CTF refinement. This resulted in reconstructions of 2.8 and 3.2 Å for the left and right halves, respectively, as evaluated by the GSFSC with a cutoff of 0.143 (Extended Data Fig. 4).

Model building and refinement

Model building was initiated by docking a previous group II RNP structure (PDB: 7UIN) into the generated reconstructions using UCSF Chimera42,43,44 (Chimera v.1.15 and ChimeraX v.1.2.5). NAMDINATOR[45] (https://namdinator.au.dk/) was used for flexible fitting of the docked models to obtain better starting models. The models were then manually rebuilt in COOT (COOT v.0.9.6) to accommodate for the changes in branch helix position, the extra ligated exons and the metal ion core. Density for the distal portion of D6 was weak in the reconstructions but this region was modelled as a helix nonetheless on the basis of data that demonstrated that this domain forms a canonical helix. The three-way junction of the D4 arm and portions of the α–α′ interaction were not modelled as the density is difficult to interpret. The final pre-1F, pre-2F and post-2F models were improved by iterative rounds of real-space refinement against the sharpened cryoEM map in PHENIX (Phenix v.1.20.1-4487) using secondary structure restraints for RNA, protein and DNA, as well Ramachandran and rotamer restraints for protein chains and subsequent rebuilding in COOT46,47,48. Model building and validation statistics are listed in Extended Data Table 1. Directional resolution anisotropy analyses were performed using the 3DFSC[23] web server (https://3dfsc.salk.edu/).

Protein conservation analysis

Group IIC maturase protein sequences were obtained from the Bacterial Group II Intron Database[49] (http://webapps2.ucalgary.ca/~groupii/). Roughly 90 sequences were aligned with ClustalOmega. Alignments were analysed and visualized using JalView.

RNA conservation analysis

Sequences corresponding to group IIC D1c and D6 sequences were obtained from the Bacterial Group II Intron Database[49] (http://webapps2.ucalgary.ca/~groupii/). Sequences were aligned with ClustalOmega and alignments were visualized and analysed with JalView.

Figure preparation

Figures and illustrations were prepared using PyMOL (PyMOL v.2.6.0), GraphPad Prism v.9.2, RNA2Drawer[50] (https://rna2drawer.app/) and Adobe Illustrator.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

All data are available in the main text and the Supplementary materials. CryoEM maps generated in this study are deposited in the Electron Microscopy Data Bank with codes EMD-40986 (pre-1F), EMD-40985 (pre-2F) and EMD-40987 (post-2F). Structural models are available in the Protein Data Bank with PDB accession codes 8T2S (pre-1F), 8T2R (pre-2F) and 8T2T (post-2F). Spliceosome and group II intron structural models used in this study (as an initial model for building or for comparison) are publicly available with the following PDB accession codes: 6J6Q (yeast spliceosome B* complex), 7B9V (yeast spliceosome C complex), 5MQ0 (yeast spliceosome C* complex), 7UIN (E. rectale group II RNP) and 4FAQ (O. iheyensis group II intron RNA).

References

1. Shi, Y. Mechanistic insights into precursor messenger RNA splicing by the spliceosome. Nat. Rev. Mol. Cell Biol. 18, 655–670 (2017).

2. Galej, W. P., Toor, N., Newman, A. J. & Nagai, K. Molecular mechanism and evolution of nuclear pre-mRNA and group II intron splicing: insights from cryo-electron microscopy structures. Chem. Rev. 118, 4156–4176 (2018).

3. Wright, C. J., Smith, C. W. J. & Jiggins, C. D. Alternative splicing as a source of phenotypic diversity. Nat. Rev. Genet. 23, 697–710 (2022).

4. Rogalska, M. E., Vivori, C. & Valcarcel, J. Regulation of pre-mRNA splicing: roles in physiology and disease and therapeutic prospects. Nat. Rev. Genet. 24, 251–269 (2023).

5. Zhao, C. & Pyle, A. M. Structural insights into the mechanism of group II intron splicing. Trends Biochem. Sci. 42, 470–482 (2017).

6. Wilkinson, M. E., Fica, S. M., Galej, W. P. & Nagai, K. Structural basis for conformational equilibrium of the catalytic spliceosome. Mol. Cell 81, 1439–1452 (2021).

7. Marcia, M. & Pyle, A. M. Visualizing group II intron catalysis through the stages of splicing. Cell 151, 497–507 (2012).

8. Zhao, C. & Pyle, A. M. Crystal structures of a group II intron maturase reveal a missing link in spliceosome evolution. Nat. Struct. Mol. Biol. 23, 558–565 (2016).

9. Galej, W. P., Oubridge, C., Newman, A. J. & Nagai, K. Crystal structure of Prp8 reveals active site cavity of the spliceosome. Nature 493, 638–643 (2013).

10. Belfort, M. & Lambowitz, A. M. Group II intron RNPs and reverse transcriptases: from retroelements to research tools. Cold Spring Harb. Perspect. Biol. 11, a032375 (2019).

11. Eickbush, T. H. Mobile introns: retrohoming by complete reverse splicing. Curr. Biol. 9, R11–R14 (1999).

12. Toor, N., Keating, K. S., Taylor, S. D. & Pyle, A. M. Crystal structure of a self-spliced group II intron. Science 320, 77–82 (2008).

13. Robart, A. R., Chan, R. T., Peters, J. K., Rajashankar, K. R. & Toor, N. Crystal structure of a eukaryotic group II intron lariat. Nature 514, 193–197 (2014).

14. Chan, R. T. et al. Structural basis for the second step of group II intron splicing. Nat. Commun. 9, 4676 (2018).

15. Qu, G. et al. Structure of a group II intron in complex with its reverse transcriptase. Nat. Struct. Mol. Biol. 23, 549–557 (2016).

16. Haack, D. B. et al. Cryo-EM structures of a group II intron reverse splicing into DNA. Cell 178, 612–623 (2019).

17. Chung, K. et al. Structures of a mobile intron retroelement poised to attack its structured DNA target. Science 378, 627–634 (2022).

18. Zhao, C., Liu, F. & Pyle, A. M. An ultraprocessive, accurate reverse transcriptase encoded by a metazoan group II intron. RNA 24, 183–195 (2018).

19. Wan, R., Bai, R., Yan, C., Lei, J. & Shi, Y. Structures of the catalytically activated yeast spliceosome reveal the mechanism of branching. Cell 177, 339–351 (2019).

20. Liu, N. et al. Exon and protein positioning in a pre-catalytic group II intron RNP primed for splicing. Nucleic Acids Res. 48, 11185–11198 (2020).

21. Dandey, V. P. et al. Spotiton: new features and applications. J. Struct. Biol. 202, 161–169 (2018).

22. Noble, A. J. et al. Reducing effects of particle adsorption to the air–water interface in cryo-EM. Nat. Methods 15, 793–795 (2018).

23. Tan, Y. Z. et al. Addressing preferred specimen orientation in single-particle cryo-EM through tilting. Nat. Methods 14, 793–796 (2017).

24. Zimmerly, S. & Semper, C. Evolution of group II introns. Mob. DNA 6, 7 (2015).

25. Liu, Q. et al. Branch-site selection in a group II intron mediated by active recognition of the adenine amino group and steric exclusion of non-adenine functionalities. J. Mol. Biol. 267, 163–171 (1997).

26. Chu, V. T., Adamidi, C., Liu, Q., Perlman, P. S. & Pyle, A. M. Control of branch-site choice by a group II intron. EMBO J. 20, 6866–6876 (2001).

27. Bertram, K. et al. Structural insights into the roles of metazoan-specific splicing factors in the human step 1 spliceosome. Mol. Cell 80, 127–139 (2020).

28. Boulanger, S. C. et al. Length changes in the joining segment between domains 5 and 6 of a group II intron inhibit self-splicing and alter 3’ splice site selection. Mol. Cell. Biol. 16, 5896–5904 (1996).

29. Fica, S. M. et al. Structure of a spliceosome remodelled for exon ligation. Nature 542, 377–380 (2017).

30. Daniels, D. L., Michels, W. J. Jr & Pyle, A. M. Two competing pathways for self-splicing by group II introns: a quantitative analysis of in vitro reaction rates and products. J. Mol. Biol. 256, 31–49 (1996).

31. Steitz, T. A. & Steitz, J. A. A general two-metal-ion mechanism for catalytic RNA. Proc. Natl Acad. Sci. USA 90, 6498–6502 (1993).

32. Genna, V., Colombo, M., De Vivo, M. & Marcia, M. Second-shell basic residues expand the two-metal-ion architecture of DNA and RNA processing enzymes. Structure 26, 40–50 (2018).

33. Liu, Y. C., Chen, H. C., Wu, N. Y. & Cheng, S. C. A novel splicing factor, Yju2, is associated with NTC and acts after Prp2 in promoting the first catalytic reaction of pre-mRNA splicing. Mol. Cell. Biol. 27, 5403–5413 (2007).

34. Pyle, A. M. Group II intron self-splicing. Annu. Rev. Biophys. 45, 183–205 (2016).

35. Zhao, C., Rajashankar, K. R., Marcia, M. & Pyle, A. M. Crystal structure of group II intron domain 1 reveals a template for RNA assembly. Nat. Chem. Biol. 11, 967–972 (2015).

36. Qin, P. Z. & Pyle, A. M. Stopped-flow fluorescence spectroscopy of a group II intron ribozyme reveals that domain 1 is an independent folding unit with a requirement for specific Mg2+ ions in the tertiary structure. Biochemistry 36, 4718–4730 (1997).

37. Guo, L. T., Olson, S., Patel, S., Graveley, B. R. & Pyle, A. M. Direct tracking of reverse-transcriptase speed and template sensitivity: implications for sequencing and analysis of long RNA molecules. Nucleic Acids Res. 50, 6980–6989 (2022).

38. Liu, T., Patel, S. & Pyle, A. M. Making RNA: using T7 RNA polymerase to produce high yields of RNA from DNA templates. Methods Enzymol. (in the press).

39. Liu, T. & Pyle, A. M. Discovery of highly reactive self-splicing group II introns within the mitochondrial genomes of human pathogenic fungi. Nucleic Acids Res. 49, 12422–12432 (2021).

40. Punjani, A., Rubinstein, J. L., Fleet, D. J. & Brubaker, M. A. cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination. Nat. Methods 14, 290–296 (2017).

41. Punjani, A., Zhang, H. & Fleet, D. J. Non-uniform refinement: adaptive regularization improves single-particle cryo-EM reconstruction. Nat. Methods 17, 1214–1221 (2020).

42. Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).

43. Goddard, T. D., Huang, C. C. & Ferrin, T. E. Visualizing density maps with UCSF Chimera. J. Struct. Biol. 157, 281–287 (2007).

44. Pettersen, E. F. et al. UCSF Chimera—a visualization system for exploratory research and analysis. J. Comput. Chem. 25, 1605–1612 (2004).

45. Kidmose, R. T. et al. Namdinator—automatic molecular dynamics flexible fitting of structural models into cryo-EM and crystallography experimental maps. IUCrJ 6, 526–531 (2019).

46. Adams, P. D. et al. PHENIX: a comprehensive Python-based system for macromolecular structure solution. Acta Crystallogr. D 66, 213–221 (2010).

47. Afonine, P. V. et al. Real-space refinement in PHENIX for cryo-EM and crystallography. Acta Crystallogr. D 74, 531–544 (2018).

48. Liebschner, D. et al. Macromolecular structure determination using X-rays, neutrons and electrons: recent developments in Phenix. Acta Crystallogr. D 75, 861–877 (2019).

49. Dai, L., Toor, N., Olson, R., Keeping, A. & Zimmerly, S. Database for mobile group II introns. Nucleic Acids Res. 31, 424–426 (2003).

50. Johnson, P. Z., Kasprzak, W. K., Shapiro, B. A. & Simon, A. E. RNA2Drawer: geometrically strict drawing of nucleic acid structures with graphical structure editing and highlighting of complementary subsequences. RNA Biol. 16, 1667–1671 (2019).

Acknowledgements

A.M.P. is an Investigator and L.X. is a Research Associate of the Howard Hughes Medical Institute (HHMI). We thank J. Peng for computational assistance and S. Devarkar for help with model building. We thank M. Llaguno (Yale) for assistance in Chameleon sample preparation. We thank S. Yang, X. Zhao and Z. Yu (HHMI Janelia) and S. Wu, J. Lin and K. Zhou (Yale) for assistance in cryoEM data collection. We thank W. Wang, S. Patel and O. Fedorova for insights and advice throughout this project. We thank M. Riggi and J. Iwasa (University of Utah) for their help with animating the group II intron splicing pathway. This work was supported by HHMI and the Gruber Foundation (Gruber Science Fellowship to K.C.). CryoEM data were collected with microscopes at the Yale CryoEM Resource Core that is funded in part by the NIH (S10OD023603).

Author information

Author notes

1. These authors contributed equally: Ling Xu, Tianshuo Liu, Kevin Chung

Authors and Affiliations

1. Howard Hughes Medical Institute, Chevy Chase, MD, USA

Ling Xu & Anna Marie Pyle

2. Department of Molecular, Cellular and Developmental Biology, Yale University, New Haven, CT, USA

Ling Xu, Tianshuo Liu & Anna Marie Pyle

3. Department of Molecular Biophysics and Biochemistry, Yale University, New Haven, CT, USA

Kevin Chung

4. Department of Chemistry, Yale University, New Haven, CT, USA

Anna Marie Pyle

Contributions

L.X., T.L., K.C. and A.M.P. conceived the project and designed the experiments. T.L., L.X. and K.C. performed biochemical purification of the sample and prepared cryoEM samples. L.X. and K.C. performed cryoEM data analysis and model building. L.X., T.L. and K.C. performed biochemical assays. L.X., K.C. and T.L. contributed to the structure analysis. L.X., T.L. and K.C. wrote the manuscript. L.X. and A.M.P. coordinated and supervised the project.

Corresponding authors

Correspondence to Ling Xu or Anna Marie Pyle.

Ethics declarations

Competing interests

A.M.P. is a cofounder of IntronX and RNAConnect. A.M.P. has patents on MarathonRT and its use as a biotechnological enzyme. The remaining authors declare no competing interests.

Peer review

Peer review information

Nature thanks Reinhard Lϋhrmann and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Biochemical investigation of group II intron branching.

The reactions are conducted under a condition that strongly favours intron branching (50 mM NH_4-HEPES pH 7.5, 500 mM NH_4Cl and 30 mM MgCl_2). a. A denaturing 5% PAGE gel showing the time course of E.r. intron self-splicing. The intron precursor RNA is radiolabeled with 32P uridine and the gel is imaged by autoradiography. Hydrolysis is the predominant splicing pathway, giving only linear intron. b. A quantitative plot of the E.r. intron self-splicing time course. Individual in vitro splicing time course with one independent isolated intron RNA and maturase sample is shown. c. A denaturing 5% PAGE gel showing the time course of E.r. intron splicing in the presence of the maturase protein (4-molar excess to the precursor RNA). With the facilitation of its cognate maturase protein, the major splicing pathway of E.r. intron is switched to branching, yielding dominantly lariat intron. Individual in vitro splicing time course with one independent isolated intron RNA and maturase sample is shown. d. A quantitative plot of the maturase-mediated E.r. intron splicing time course. The apparent precursor depletion rate increases from 0.03 min−1 in the case of self-splicing to 0.16 min−1 with maturase assistance. Gel filtration chromatogram of (e) the pre-1F and pre-2F and (f) the post-2F RNP sample preparation. e. The two peaks correspond to aggregates, co-eluted pre-1F and pre-2F RNP sample respectively. f. The two peaks arise from the post-2F RNP sample and the spliced exons respectively. A_260 and A_280 traces are shown in red and blue respectively.

Extended Data Fig. 2 CryoEM Workflow for the Pre-1F and Pre-2F RNPs.

CryoEM data processing workflow of the pre-1F and pre-2F RNP samples (details in methods).

Extended Data Fig. 3 CryoEM Validation for the Pre-1F and Pre-2F RNPs.

a. Local resolution and particle distribution of the pre-1F overall and the locally refined pre-1F (left) and pre-1F (right) maps. b. Local resolution and particle distribution of the pre-2F overall and the locally refined pre-2F (left) and pre-2F (right) maps. c. 3DFSC analysis of the pre-1F overall, the focused refined pre-1F (left) and the focused refined pre-1F (right) reconstructions. d. 3DFSC analysis of the pre-2F overall, the focused refined pre-2F (left) and the focused refined pre-2F (right) reconstructions. e. FSC curves with the gold standard threshold of 0.143 for the three pre-1F RNP maps. f. Map to model FSC curve for the pre-1F model refined against the pre-1F overall map. g. FSC curves with the gold standard threshold of 0.143 for the three pre-2F RNP maps. h. Map to model FSC curve for the pre-2F model refined against the pre-2F overall map.

Extended Data Fig. 4 CryoEM Workflow for the Post-2F RNP.

a. CryoEM data processing workflow for the post-2F RNP (details in methods). b. Local resolution and particle distribution of the post-2F overall and the locally refined post-2F (left) and post-2F (right) maps. c. FSC curves with the gold standard threshold of 0.143 for the three post-2F RNP maps. d. Map to model FSC curve for the post-2F model refined against the post-2F overall map. e. 3DFSC analysis of the post-2F overall, the focused refined post-2F (left) and the focused refined post-2F (right) reconstructions.

Extended Data Fig. 5 Secondary Structure of the E.r. Group IIC Intron and Conservation of Protein and RNA Sequences.

a. Secondary structure diagram of the E.r. IIC intron. RNA domains are labelled D1-D6 (four subdomains of D1 are labelled as D1a, D1b, D1c and D1d respectively). Tertiary interactions are labelled with Greek letters. Exon binding sequences and intron binding sequences are labelled as EBS and IBS respectively. b. Sequence alignment of the D1c and D6 regions of bacterial group IIC introns. c. Sequence alignment of bacterial group IIC maturases. The regions involved in interactions with D6 within the thumb and DBD are shown under the arrows.

Extended Data Fig. 6 Mechanistic insights into 5′ Splice Site Recognition and Branch Helix Hinge Motions.

Comparison of the interactions surrounding the conserved 5′ splice site GU nucleotides for a. the group II RNP in the pre-1F state and b. the yeast spliceosome C complex (PDB:7B9V). The position of bpA adjacent nucleotides (C630 and U631 in a. and U68 and A69 in b.) are indicated with dashed lines and drawn as cartoon batons. c. Interaction network between J5/6 and J4/5. J5/6 RNA phosphate backbone conformations in d. the pre-1F and e. pre-2F states.

Extended Data Fig. 7 Comparison of the Heteronuclear Metal Ion Core.

a. The catalytic elements including the 5′ splice site, branchpoint A, J2/3 linker, catalytic triad, two-nt bulge and the metal ions (M1, M2, K1 and K2 equivalents) for the pre-1F RNP are shown. Density from the cryoEM map of the corresponding regions is shown to indicate model fit. Catalytic core of the b. O.i. intron (PDB: 4FAQ) and c. the spliceosome C complex (PDB:7B9V) and the key mechanistic elements as in (a) are displayed. The attacking water nucleophile is shown as a light blue sphere in (b).

Extended Data Fig. 8 Comparison of Protein-Branch Helix Interactions.

a. Interactions between the DBD and thumb of the maturase protein with D6 in the group II intron. G1 and bpA (A632) are shown in sticks. Lys361 is shown interacting with G1. Surface charges around the protein are shown (blue-positive, white-neutral and red-negative). b. Interactions between the thumb (in red) and linker (in yellow) of Prp8 with the U2 snRNA-intron branch helix in the yeast spliceosome B* complex (PDB: 6J6Q). Surface charges of the proteins are shown with the same colour code as in (a). G1 and bpA (A70) are shown in sticks. Arg3 from the Yju2 protein (in green) is shown interacting with G1.

Extended Data Fig. 9 Model of Group II RNP Life cycle.

In the first step of splicing, the 5′-exon (in black) is recognized and juxtaposed against the branchpoint A (shown as a star) of the D6 helix. The branching reaction occurs, forming the lariat bond (shown as a dash) and the D6 helix disengages from D1c and the thumb and DBD of the maturase protein, swinging downwards and bringing the 3′-exon into the active site. The 5′-splice site is now poised to attack the scissile phosphate of the 3′-exon, splicing the exons together. After the first two steps of splicing, the ligated exon is released and the apoRNP functions as a retroelement, undergoing retrotransposition, TPRT and transcription to complete the group II intron life cycle.

Extended Data Table 1 CryoEM Data Collection, Refinement and Validation Statistics

Supplementary information

Supplementary Figures

Supplementary Fig. 1: CryoEM sample preparation denaturing gel from Fig. 1d. a, Various splicing conditions were used to obtain samples that contained the prebranching, preligation and postligation complexes. b, An SDS gel showing the protein contents of the same samples in bottom inset of a. Boxed regions indicate the cropped images shown in the main text figure. c, Splicing assay denaturing gels shown in Fig. 2d,h. All four replicates of the gels are shown. Boxed regions indicate the cropped images shown in the main text figure. Supplementary Fig. 2: Splicing gel time courses in the absence or presence of WT maturase protein from Extended Data Fig. 1a,c. Boxed regions indicate the cropped images shown in the Extended Data Fig. 1.

Reporting Summary

Supplementary Video 1

Interactions holding D6 in the up position. Contacts with D1 and the maturase that arrange D6 before branching.

Supplementary Video 2

Local movements of the branchpoint A. Motion of the branchpoint A during splicing.

Supplementary Video 3

Conformational dynamics of the branch helix. Large-scale movements of D6 during the transition from branching to exon ligation.

Supplementary Video 4

Animation of the structures of group II intron RNP undergoing forward splicing. Animation of group II intron RNP assembly, catalytic and dynamical mechanisms during the two steps of the forward splicing reaction.

Supplementary Video 5

Animation of the group II intron RNP life cycle. Animation of the full life cycle of the group II intron RNP from splicing to retrohoming into new genomic loci.

Peer Review File

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Xu, L., Liu, T., Chung, K. et al. Structural insights into intron catalysis and dynamics during splicing. Nature 624, 682–688 (2023). https://doi.org/10.1038/s41586-023-06746-6

- Received: 17 July 2023

- Accepted: 13 October 2023

- Published: 22 November 2023

- Issue Date: 21 December 2023

- DOI: https://doi.org/10.1038/s41586-023-06746-6

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Sorry, a shareable link is not currently available for this article.

Copy to clipboard

Provided by the Springer Nature SharedIt content-sharing initiative

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Cryo-EM reveals dynamics of Tetrahymena group I intron self-splicing

Article 16 March 2023

Bingnan Luo, Chong Zhang, … Zhaoming Su

Visualizing group II intron dynamics between the first and second steps of splicing

Article Open access 05 June 2020

Jacopo Manigrasso, Isabel Chillón, … Marco Marcia

Snapshots of the second-step self-splicing of Tetrahymena ribozyme revealed by cryo-EM

Article Open access 16 March 2023

Shanshan Li, Michael Z. Palo, … Kaiming Zhang

Table of Contents

Nature: 21 Dec, 2023

This Week

End the glaring inequity in international science collaborations

From Einstein to AI: how 100 years have shaped science

The postdoc experience is broken. Funders such as the NIH must help to reimagine it

Research prizes are opaque and rife with bias — it’s time to shake them up

News in Focus

More than 10,000 research papers were retracted in 2023 — a new record

‘Biocomputer’ combines lab-grown brain tissue with electronic hardware

How a bullying scandal closed a historic astronomy department

‘It’s all gone’: CAR-T therapy forces autoimmune diseases into remission

COP28 climate summit signals the end of fossil fuels — but is it enough?

RNA biologist loses disability case against Howard Hughes Medical Institute

The best science images of 2023 — Nature’s picks

Nature’s 10: ten people (and one non-human) who helped shape science in 2023

The engineer who helped India to reach the Moon

Amazon protector: the Brazilian politician who turned the tide on deforestation

Making mice with two dads: this biologist rewrote the rules on sexual reproduction

Nuclear-fusion breakthrough: this physicist helped to achieve the first-ever energy gain

How Earth’s first global heat officer is tackling climate change

OpenAI’s chief scientist helped to create ChatGPT — while worrying about AI safety

Superconductivity debunker: this physicist exposed flaws in a blockbuster claim

Weight-loss-drug pioneer: this biochemist finally gained recognition for her work

Malaria fighter: this researcher paved the way for a game-changing vaccine

The cancer physician who helped to deliver a life-extending treatment

ChatGPT and science: the AI system was a force in 2023 — for good and bad

Books & Arts

Black holes, love and poetry — an artistic exploration of intimacy and adventure

Prebunking propaganda, and how to live well: 2023’s best of Books in brief

Does quantum theory imply the entire Universe is preordained?

Opinion

To build a better world, stop chasing economic growth

Devise an ethical open-access publishing model

Work

The scientific workplace in 2023

How I’m protecting Clanwilliam sandfish

Research

2D materials ratchet up biorealism in computing

Homogenizing out-of-plane cation composition in perovskite solar cells

Autonomous chemical research with large language models

Biological carbon pump estimate based on multidecadal hydrographic data

Human mobility networks reveal increased segregation in large cities

Indigenous Australian genomes show deep structure and rich novel variation

The landscape of genomic structural variation in Indigenous Australians

Lung dendritic-cell metabolism underlies susceptibility to viral infection in diabetes

Structural insights into intron catalysis and dynamics during splicing