From Stuttgart’s first industrial revolution to Dubai’s fifth – the need for research to connect outside the academy

In the late 19th century, Stuttgart was booming. The southern German city was famously the cradle of an emerging automobile sector and had already established itself as an industrial powerhouse and centre for toolmaking, mechanical engineering and textiles. Rail connections in the Baden-Württemberg region accelerated development, transported workers and spread wealth.

One might think, then, that an obvious place for the nascent railways to reached out to would have been the historic university town of Tübingen, about 20 miles from Stuttgart. No so, Tilman Wörtz of the university’s communications department informed me, on a recent visit. In fact, explained Wörtz, an accomplished journalist, the story goes that the academic grandees of the era resisted a connection with the emerging financial and industrial powerhouse, perhaps regarding it as somewhat uncouth and vulgar to distract from deep cultural and scientific considerations to engage with the forces of commerce. So for a long time, the proposed railroad hit the buffers.

Fortunately today, thanks to the efforts of university leaders, the institution strives to connect, both with industry and the wider community. There is now a railway station, and I was thrilled to speak with a number of academics about relaying their research and knowledge to non-academic readers. Indeed, this fascinating read on rapper Haftbefehl, who is the subject of a Netflix documentary gripping Germany has already come out of the sessions, and do stay tuned in the coming weeks and months for more from the University of Tübingen, which was founded in 1477 and is now the first German member institution of The Conversation.

Fast forward a week, and I found myself in the eye of what some cast as a fourth or even fifth industrial revolution, in Dubai, incorporating AI, nanobiology, and bioengineering. The city is pitching itself as being at the heart of, and a driving force in, this new era of change, which sees civic government enabling human and technological collaboration tackling societal issues and powering growth.

For more than a decade, what is now called Protoypes For Humanity has been an exhibition at the heart of this city’s dash for development, powering projects that bring the prospect of solutions to challenges in the environment, energy, health, technology and other spheres.

When I attended Prototypes a year ago, it was still largely a showcase for PhD candidates’ projects from some of the world’s leading universities, many of which are members of The Conversation. In the last 12 months, however, a new element has been developed, under the guidance of Naren Barfield, former Provost and Deputy Vice Chancellor of the UK’s Royal College of Art. This sees senior academics come to the city to deliver papers drawing on key aspects of their research.

Full transparency, I served on the selection panel Professor Barfield designed to finalise the programme and The Conversation was a media partner for the 2025 Prototypes event.

The themes for the year were as follows:

  • Wellbeing and health futures
  • Sustainable and resilient infrastructure
  • Artificial intelligence and augmented intelligence
  • Environmental sustainability and climate action
  • Socio-economic empowerment and innovation
  • Open and speculative categories

Following short paper presentations in the Socio-Economic Empowerment category, Barfield explained the thinking behind the new element of Prototypes and the opportunity for researchers:

We are bringing together some of the world’s sharpest minds and most innovative researchers to tackle challenges faced in different parts of the planet. Dubai and this initiative provide a unique chance to generate ideas across a range of academic disciplines that might not otherwise collaborate in such an impactful way.

The Prototypes for Humanity initiative and the relatively new Professors’ Programme has a proven track record of connecting academia with policymakers, industry, and the public in a way often described elsewhere as aspirational. Here, it is actually happening.

The reference to industry struck a chord, perhaps given that I’d so recently heard that story of detachment from 19th century Stuttgart, but also because it’s a grumble I regularly encounter across the world when it comes to academia and its engagement (or lack of) beyond the university sector today.

At the conference venue, in the Emirates Towers of Dubai International Financial District, Tadeu Baldani Caravieri, director of Prototypes for Humanity, discussed the thinking behind the project and potential routes forward.

At Protoypes we’ve seen how researchers can directly drive innovation in partnership with industry and, in the case of Dubai, with the city government as a facilitator.

This has been possible thanks to some of the advantages of this state and region. But these are solutions that can, and do present wider benefits – in some cases globally relevant solutions solutions.

He later added:

This edition [of Prototypes] helped to confirm fundamental assumptions for the space we operate in, i.e. creating bridges between academic ingenuity and real-world needs. The main one is that, although there is sometimes a disconnect between university innovation capabilities and industry needs, there is genuine interest, across all of the parts in this equation, to overcome obstacles and do more. We have enabled and witnessed very promising and results-oriented conversations between academia and potential partners, from PhDs and private sector discussing pilots in applied robotics, to professors supporting a humanitarian agency to rethink aid allocation systems, to multinationals looking to fuel their R&D roadmaps.

Dubai is an excellent incubator for these bridges we are building but, in keeping with the city’s outlook and spirit, we want to enable impact across the world – so it’s just natural that, in the future, we hope to open structured avenues for multi-city collaborations, where local ecosystems complement each other’s strengths.

Prototypes’ community brings in research talent from more than 800 universities around the world, including many academics who have also engaged with The Conversation. For instance, Jeremy Howick, of the University of Leicester, presented on empathy in healthcare in the age of AI, and has written this account. Further articles based on projects that exhibited and on the professors’ papers will be published on The Conversation and will be accessible via this link.

Stay tuned to read more on critical and diverse research relating to subjects such as monitoring and diagnosing pre-eclampsia (Patricia Maguire, University College Dublin, using seaweed to create a sustainable packaging alternative (Austeja Platukyte, Vilnius Academy of Arts ) and the emergent Internet of Beings (Francesco Grillo, Bocconi University, Milan).

The Conversation

file-20251121-56-gsvi04.jpg

Colleges teach the most valuable career skills when they don’t stick narrowly to preprofessional education

Tracking graduates’ earnings is just one way to measure the benefit of higher education. iStock/Getty Images Plus

Across state legislatures and in Congress, debates are intensifying about the value of funding certain college degree programs – and higher education, more broadly.

The growing popularity of professional graduate degrees over the past several decades – including programs in business administration and engineering management – has reshaped the economics of higher education. Unlike traditional academic graduate programs, which are often centered on research and scholarship, these professionally oriented degrees are designed primarily for workforce advancement and typically charge much higher tuition.

These programs are often expensive for students and are sometimes described as cash-cow degrees for colleges and universities, because the tuition revenue far exceeds the instructional costs.

Some universities and colleges also leverage their brands to offer online, executive or certificate-based versions of these programs, attracting many students from the U.S. and abroad who pay the full tuition. This steady revenue helps universities subsidize tuition for other students who cannot pay the full rate, among other things.

Yet a quiet tension underlies this evolution in higher education – the widening divide between practical, technical training and a comprehensive education that perhaps is more likely to encourage students to inquire, reflect and innovate as they learn.

An overlooked factor

Some states, including Texas, track salary data for graduates of every program to measure worth through short-term earnings. This approach may strike many students and their families as useful, but I believe it overlooks a part of what makes higher education valuable.

A healthy higher education system depends not only on producing employable graduates but also on cultivating citizens and leaders who can interpret uncertainty, question assumptions and connect ideas across disciplines.

When assessing disciplines such as English, philosophy, history and world languages, I think that we should acknowledge their contributions to critical thought, communication and ethical reasoning.

These academic disciplines encourage students to synthesize ideas, construct arguments and engage in meaningful debate. Some law schools often draw their strongest students from these backgrounds because they nurture analytical and rhetorical skills essential for navigating complex civic and legal issues.

Historically, poets and writers have often been among the first to be silenced by authoritarian regimes. It’s a reminder of the societal power of inquiry and expression that I believe higher education should protect.

A group of young people wear white jackets and stand around a dummy dressed with a pink blanket over it in a hospital bed.
Undergraduate students who want to become doctors or work in other specialized fields are often encouraged to take only classes that connect with their long-term career trajectory.
Glenn Beil/Florida A&M University via Getty Images

Why students stay on narrow professional paths

Students entering college today face significant pressure to choose what they might see as safe majors that will result in a well-paying career. For aspiring physicians and engineers, the path is often scripted early by steering them toward physical and biosciences. High test scores, internships and other stepping stones are treated as nonnegotiable. Parents and peers can reinforce this mindset.

Most colleges and universities do not reward a future medical student who wants to major in comparative literature, or an engineering student who is spending time on philosophy.

Students’ majors also typically place course requirements on them, in addition to a school’s general course requirements. This often does not leave a lot of room for students to experiment with different classes, especially if they are pursuing vocationally focused majors, such as engineering.

As a result, I’ve seen many students trade curiosity for credentialing, believing that professional identity must come before intellectual exploration.

As someone who began my education in psychology and later transitioned into engineering, I have seen how different intellectual traditions approach the same human questions. Psychology teaches people to observe behavior and design experiments. Engineering trains students to model systems and optimize performance.

When combined, they help reveal how humans interact with technology and how technological solutions reshape human behavior.

In my view, these are questions neither field can answer alone.

Initiative is the missing ingredient

One of the most important and often overlooked ingredients in thriving high tech, medical and business environments is initiative. I believe students in the humanities routinely practice taking initiative by framing questions, interpreting incomplete information and proposing original arguments. These skills are crucial for scientific or business innovation, but they are often not emphasized in structured science, technology, engineering and mathematics – or STEM – coursework.

Initiative involves the willingness to move first and to see around corners, defining the next what-if, rallying others and building something meaningful even when the path is uncertain.

To help my engineering students practice taking initiative, I often give them deliberately vague instructions – something they rarely experience in their coursework. Many students, even highly capable ones, hesitate to take initiative because their schooling experience has largely rewarded caution and compliance over exploration. They wait for clarity or for permission – not because they lack ability, but because they are afraid to be wrong.

Yet in business, research labs, design studios, hospitals and engineering firms, initiative is the quality employers most urgently need and cannot easily teach. Broader educational approaches help cultivate this confidence by encouraging students to interpret ambiguity rather than avoid it.

How teaching can evolve

Helping all students develop a sense of initiative and innovation requires university leaders to rethink what success looks like.

Universities can begin with achievable steps, such as rewarding cross-disciplinary teaching and joint appointments in promotion and tenure criteria.

At the University of Iowa’s Driving Safety Research Institute, where our teams blend engineering, medicine, public health and psychology, students quickly learn that a safe automated vehicle is not just a technical system but also a behavioral one. Understanding how human drivers respond to automation is as important as the algorithms that govern the vehicle.

Other institutions are modeling this approach of integrating social, behavioral and physical sciences.

Olin College of Engineering, a school in Needham, Massachusetts, builds every project around both technical feasibility and human context. Courses are often co-taught by humanities and engineering professors, and projects require students to articulate not only what they built but why it matters.

Still, integrating liberal and technical education is difficult in practice. Professional curricula often overflow with accreditation requirements. Faculty incentives reward specialization more than collaboration. Students and parents, anxious about debt and job security, hesitate to spend credits outside of a student’s major.

Rethinking what success means

I believe that higher education’s purpose is not to produce uniform workers but adaptable thinkers.

It might not be productive to center conversations about defending the liberal arts or glorifying STEM. Rather, I think that people’s focus should be on recognizing that each field is incomplete without the other.

Education for a complex world must cultivate depth, initiative and perspective. When students connect disciplines, question assumptions and act with purpose, they are prepared not only for their first job but for a lifetime of learning and leadership.

The Conversation

Daniel V. McGehee does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

file-20251117-56-svciav.jpg

Australian businesses have actually been slow to adopt AI, survey finds

Burst/Unsplash

Over the past few years, we have repeatedly been told artificial intelligence (AI) is coming for our jobs.

In May last year, the International Monetary Fund warned the technology was hitting labour markets like a “tsunami”. Many of the tech giants behind the technology have been making their own confident predictions about which jobs will be wiped out first.

Yet inside many Australian firms, the reality is much less dramatic.

Last week, the Reserve Bank of Australia released the findings from a 2025 survey of 100 medium and large-sized firms, which sought to understand how technology investments have been affecting the way businesses run. It found enterprise-wide AI transformation was the exception rather than the norm.

This presents a strange mismatch: a loud global story about an AI “jobpocalypse”, and a much quieter story inside firms about experiments, pilots and a lot of waiting around for real productivity gains to show up. The Reserve Bank’s report helps explain why.

Laying the groundwork

Australian firms haven’t been ignoring AI and technology. The report notes that over the past decade, total IT investment has grown by almost 80%, faster than other types of capital investment.

Much of that has gone into software, cybersecurity, cloud and upgrading internal systems such as customer relationship management and enterprise resource planning platforms.

While these investments are mostly about keeping the lights on and do not deliver big productivity pay offs on their own, they are essential groundwork to make firms’ systems and processes ready for AI.

‘Minimal’ use of AI

AI is now starting to move up the priority list. About two-thirds of firms surveyed said they have adopted AI “in some form”.

But for the largest group – representing nearly 40% of all respondents – this use was still “minimal”.

Man typing on laptop, screen open to ChatGPT website
Many firms reported only ‘minimal’ use of AI tools.
Viralyft/Pexels

The most common use cases were tasks such as summarising emails or drafting text using off-the-shelf-products like Microsoft Copilot or ChatGPT.

Just over 20% of all firms reported “moderate” adoption, using AI to assist with tasks such as demand forecasting or inventory management.

And a small frontier group – less than 10% of all firms – said they had embedded AI into more advanced processes such as fraud detection.

Impact on jobs

Asked about jobs and the future, companies said AI and automation would likely save some labour, particularly on traditionally time-consuming tasks. But they didn’t expect it to cause catastrophic job losses.

Around half said they expected AI and machine learning would lead to a modest reduction in headcount over the next three years.

Firms planning to reduce their headcount expected to do so through natural attrition (employees resigning or retiring), lower intake of new staff, and redundancies.

Routine finance, administration and contact centre work were seen as most at risk of being automated. But firms also said AI would likely create demand for new roles related to the technology – such as in cybersecurity and redesigning processes.

Why are Australian firms so slow?

The Reserve Bank’s findings align with other reports showing Australia as a cautious adopter of AI when compared, for example, to the United States.

Global report cards on AI adoption and innovation more broadly consistently place Australia behind many other advanced economies.

A few key themes stand out from the Reserve Bank’s survey and other international evidence.

First, much of Australia’s recent technology investment has gone into cybersecurity, compliance, legacy system upgrades, data quality improvements and cloud migration. This is a necessary first step before AI investments.

Second, many firms have been struggling to hire the skills needed to drive AI transformation, such as data engineers and data scientists.

And third, Australia’s business culture is cautious, with low trust and high levels of concern about AI. Adding to this, individuals’ “shadow” use of AI tools (without telling their bosses) can mask the true extent of the technology’s adoption.

Do Australian firms need more support?

If Australian businesses want to move beyond “ChatGPT for emails”, evidence points to several practical steps they and governments can make.

Better engagement of company boards is crucial for moving beyond shallow pilots of digital technology, especially when workers distrust AI and directors see it as a risk. Providing evidence of more successful use cases is an important part of this shift.

Australia also needs to invest in lifting AI skills across the workforce, with a particular focus on reskilling workers and preparing for the likely decline in entry-level jobs for young people.

The report indicates firms see the uncertain regulatory environment as a major barrier. The goal should be to have clear, risk-based rules that make safe experimentation easier.

Without these, firms are effectively told two things at once: “move fast to stay competitive” and “don’t you dare breach privacy, copyright or ethics”.




Read more:
AI ‘workslop’ is creating unnecessary extra work. Here’s how we can stop it


Brace for a bumpy ride

It appears we may not yet be racing toward a world without workers, but rather a messy and uneven transition.

The number of AI firms in Australia has grown substantially, and there is clear momentum and optimism.

For now, however, many businesses are still working out how to make AI useful, embed it into workflows and manage its risks. Their biggest complaints are about ambiguous regulation and skills shortages – not having too many humans.

The Conversation

Stan Karanasios does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

file-20251106-64-pgubjv.jpg

Trump’s tariffs threaten the future of innovation – and UK tech could be collateral damage

Gorodenkoff/Shutterstock

US president Donald Trump’s 15% baseline tariffs on EU imports may read like a throwback to old-school protectionism, designed to safeguard American jobs and manufacturing. But in today’s globalised and digitally driven economy, the risk isn’t just to steel or car factories, it’s to innovation itself.

The world’s most advanced technologies rely on complex, deeply integrated supply chains. Evidence from 2023 shows that even temporary US tariff shocks disrupted relationships between firms. And these tariffs won’t just hit the EU. They will disrupt the high-value tech ecosystems of partners like the UK – especially firms contributing to artificial intelligence (AI), semiconductor design and cybersecurity.

These industries underpin national resilience, data security and the competitiveness of advanced economies. For the UK, which often positions itself as a global innovation hub post-Brexit, the fallout could be significant.

Take ARM Holdings, the Cambridge-based semiconductor giant whose chip designs power 99% of the world’s smartphones and an increasing share of AI infrastructure.

ARM doesn’t manufacture chips itself. Instead, it licenses its architecture to firms like Apple, Nvidia and Qualcomm. That makes it a prime example of the UK’s value in the global innovation chain: high intellectual property (IP), low carbon footprint, huge reach.

ARM’s position as a vital link in the supply chain underlines another point. Trade policy aimed at traditional manufacturing sectors can inadvertently destabilise tech-intensive, IP-led sectors like semiconductors and software. This is echoed in research examining global tariff spillovers on tech competitiveness.

If tariffs are applied to components or design work linked to traded goods that cross EU or UK borders en route to US manufacturers, it introduces a layer of risk and cost to innovative firms and their global partners.

Even if a company’s work isn’t directly taxed, the uncertainty and red tape may make US firms think twice about sourcing from outside US jurisdictions. While Trump might present that as a victory for American manufacturing, in reality it could raise costs for US producers, damage innovation and make US firms less competitive in the industries he aims to protect.

It’s not just the giants at risk. In the UK, Cambridge’s wider tech cluster, sometimes called “Silicon Fen”, is home to dozens of ambitious AI firms. With operations spanning the UK, EU and US, companies like this depend on fast, flexible and trusted international partnerships to develop, deploy and refine their products. Tariff-related disruptions make collaboration harder at a time when speed is a competitive advantage.

This is not hypothetical. Tariffs reduce access to large markets – and when markets shrink, firms reduce investment in research and innovation.

What Trump gets wrong

Trump’s broader narrative suggests tariffs can bring back jobs and restore industrial power to the US. But innovation doesn’t work like that. A semiconductor isn’t made in one place. A cybersecurity system isn’t built by a single team. These are networked, iterative processes, involving researchers, suppliers, data centres and talent pools across continents. Disrupt that flow and you slow progress.

The UK is especially exposed because of its unique post-Brexit positioning. It trades independently from the EU but is still tightly intertwined with it, particularly in tech sectors.

Many UK firms use EU distribution centres to reach the US market or collaborate with EU partners on joint projects involving data, hardware or software This reflects the fact that the UK remains tightly integrated into European supply and value chains – exporting £358 billion of goods and services to the EU in 2024 alone. Tariffs targeting the EU could easily catch UK-originated components or design work as collateral damage.

Modelling has shown that Trump’s proposed tariffs could reduce EU-US trade volumes across multiple sectors, particularly in tech, where integrated production routes are standard.

Small and medium-sized enterprises and startups may find themselves most vulnerable. These firms typically can’t absorb sudden cost increases or legal complexities. Nor can they easily switch suppliers or reroute through different customs zones.

If you’re an early-stage AI company relying on a specific chip from Germany and a US cloud partner to train your model, a 15% tariff adds months of delays and thousands of pounds in costs, just to maintain the status quo.

From a policy perspective, the impact goes deeper. The UK government has championed sectors like AI, fintech and clean tech as pillars of economic growth. But these industries are only as strong as the networks that sustain them. If global fragmentation accelerates, the UK risks losing its role as a bridge between the US and the EU.

Meanwhile, countries like China continue to invest heavily in consolidating their innovation supply chains, from chip manufacturing to AI research, particularly in efforts to secure domestic control over advanced technologies and semiconductors. This is something that the US and EU have only recently begun to coordinate on.

In the short term, Trump’s tariff strategy may boost US customs revenue, which is up US$50 billion (£38 billion) a month by some estimates.

But this is not “free money”. These revenues are largely absorbed by businesses and ultimately passed on to consumers through higher prices, or to smaller suppliers through squeezed profit margins.

More fundamentally, it represents a belief that economic strength comes from protection rather than connection. But innovation has never worked that way. It thrives on collaboration, trust and scale. Tariffs may be politically effective, but economically they are the equivalent of building firewalls between teams that are supposed to be co-writing the future.

As the UK charts its post-Brexit global role, aligning itself with open, innovation-driven economies should be a priority. That means standing up for the integrity of global tech supply chains and recognising that disruption to one part of the system can reverberate far beyond its intended target.

The Conversation

Matthew Allen is affiliated with The Conservative Party as a party member. I am not a councillor or an MP. I am also not active in any campaigning.

file-20251105-56-z6vgyb.jpg

New technologies like AI come with big claims – borrowing the scientific concept of validity can help cut through the hype

Closely examining the claims companies make about a product can help you separate hype from reality. Flavio Coelho/Moment via Getty Images

Technological innovations can seem relentless. In computing, some have proclaimed that “a year in machine learning is a century in any other field.” But how do you know whether those advancements are hype or reality?

Failures quickly multiply when there’s a deluge of new technology, especially when these developments haven’t been properly tested or fully understood. Even technological innovations from trusted labs and organizations sometimes result in spectacular failures. Think of IBM Watson, an AI program the company hailed as a revolutionary tool for cancer treatment in 2011. However, rather than evaluating the tool based on patient outcomes, IBM used less relevant measures – possibly even irrelevant ones, such as expert ratings rather than patient outcomes. As a result, IBM Watson not only failed to offer doctors reliable and innovative treatment recommendations, it also suggested harmful ones.

When ChatGPT was released in November 2022, interest in AI expanded rapidly across industry and in science alongside ballooning claims of its efficacy. But as the vast majority of companies are seeing their attempts at incorporating generative AI fail, questions about whether the technology does what developers promised are coming to the fore.

Black screen with IBM Watson logo on a Jeopardy stand with $1,200 stood between two contestants with $0 each
IBM Watson wowed on Jeopardy, but not in the clinic.
AP Photo/Seth Wenig

In a world of rapid technological change, a pressing question arises: How can people determine whether a new technological marvel genuinely works and is safe to use?

Borrowing from the language of science, this question is really about validity – that is, the soundness, trustworthiness and dependability of a claim. Validity is the ultimate verdict of whether a scientific claim accurately reflects reality. Think of it as quality control for science: It helps researchers know whether a medication really cures a disease, a health-tracking app truly improves fitness, or a model of a black hole genuinely describes how it behaves in space.

How to evaluate validity for new technologies and innovations has been unclear, in part because science has mostly focused on validating claims about the natural world.

In our work as researchers who study how to evaluate science across disciplines, we developed a framework to assess the validity of any design, be it a new technology or policy. We believe setting clear and consistent standards for validity and learning how to assess it can empower people to make informed decisions about technology – and determine whether a new technology will truly deliver on its promise.

Validity is the bedrock of knowledge

Historically, validity was primarily concerned with ensuring the precision of scientific measurements, such as whether a thermometer correctly measures temperature or a psychological test accurately assesses anxiety. Over time, it became clear that there is more than just one kind of validity.

Different scientific fields have their own ways of evaluating validity. Engineers test new designs against safety and performance standards. Medical researchers use controlled experiments to verify treatments are more effective than existing options.

Researchers across fields use different types of validity, depending on the kind of claim they’re making.

Internal validity asks whether the relationship between two variables is truly causal. A medical researcher, for instance, might run a randomized controlled trial to be sure that a new drug led patients to recover rather than some other factor such as the placebo effect.

External validity is about generalization – whether those results would still hold outside the lab or in a broader or different population. An example of low external validity is how many early studies that work in mice don’t always translate to people.

Construct validity, on the other hand, is about meaning. Psychologists and social scientists rely on it when they ask whether a test or survey really captures the idea it’s supposed to measure. Does a grit scale actually reflect perseverance or just stubbornness?

Finally, ecological validity asks whether something works in the real world rather than just under ideal lab conditions. A behavioral model or AI system might perform brilliantly in simulation but fail once human behavior, noisy data or institutional complexity enter the picture.

Across all these types of validity, the goal is the same: ensuring that scientific tools – from lab experiments to algorithms – connect faithfully to the reality they aim to explain.

Evaluating technology claims

We developed a method to help researchers across disciplines clearly test the reliability and effectiveness of their inventions and theories. The design science validity framework identifies three critical kinds of claims researchers usually make about the utility of a technology, innovation, theory, model or method.

First, a criterion claim asserts that a discovery delivers beneficial outcomes, typically by outperforming current standards. These claims justify the technology’s utility by showing clear advantages over existing alternatives.

For example, developers of generative AI models such as ChatGPT may see higher engagement with the technology the more it flatters and agrees with the user. As a result, they may program the technology to be more affirming – a feature called sycophancy – in order to increase user retention. The AI models meet the criterion claim of users considering them more flattering than talking to people. However, this does little to improve the technology’s efficacy in tasks such as helping resolve mental health issues or relationship problems.

AI sycophancy can lead users to break relationships rather than repair them.

Second, a causal claim addresses how specific components or features of a technology directly contribute to its success or failure. In other words, it is a claim that shows researchers know what makes a technology effective and exactly why it works.

Looking at AI models and excessive flattery, researchers found that interacting with more sycophantic models reduced users’ willingness to repair interpersonal conflict and increased their conviction of being in the right. The causal claim here is that the AI feature of sycophancy reduces a user’s desire to repair conflict.

Third, a context claim specifies where and under what conditions a technology is expected to function effectively. These claims explore whether the benefits of a technology or system generalize beyond the lab and can reach other populations and settings.

In the same study, researchers examined how excessive flattery affected user actions in other datasets, including the “Am I the Asshole” community on Reddit. They found that AI models were more affirming of user decisions than people were, even when the user was describing manipulative or harmful behavior. This supports the context claim that sycophantic behavior from an AI model applies across different conversational contexts and populations.

Measuring validity as a consumer

Understanding the validity of scientific innovations and consumer technologies is critical for scientists and the general public. For scientists, it’s a road map to ensure their inventions are rigorously evaluated. And for the public, it means knowing that the tools and systems they depend on – such as health apps, medications and financial platforms – are truly safe, effective and beneficial.

Here’s how you can use validity to understand the scientific and technological innovations happening around you.

Because it is difficult to compare every feature of two technologies against each other, focus on which features you value most from a technology or model. For example, do you prefer a chatbot to be accurate or better for privacy? Examine claims for it in that area, and check that it is as good as claimed.

Consider not only the types of claims made for a technology but also which claims are not made. For example, does a chatbot company address bias in its model? It’s your key to knowing whether you see untested and potentially unsafe hype or a genuine advancement.

By understanding validity, organizations and consumers can cut through the hype and get to the truth behind the latest technologies.

The Conversation

The authors do not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.