Artificial intelligence (AI) is revolutionising healthcare, offering transformative potential in diagnostics, treatment planning, and patient care. However, its rapid integration into health practice presents significant legal, regulatory, and ethical challenges. This article explores emerging technologies and provides insights into the current regulatory landscape and key risk areas associated with AI in healthcare.
The use of AI in healthcare
Current uses
Australian healthcare settings have already begun to implement AI into daily practice,1 including tools that:
- listen to patient consultations and automatically generate notes, care plans, and orders for tests
- combine information about diseases and treatment pathways and suggest diagnoses based on patient symptoms
- analyse images and photographs to detect skin, cervical, breast, and prostate cancer
- assist surgeons in making decisions and operating instruments during surgery
- analyse risks to patients to predict bed usage or surges in the healthcare system, and
- provide companionship and monitoring in aged care settings.
Emerging technologies
Development of AI-assisted tools is also progressing at a rapid rate. Some of the emerging technologies in this space include:
- OpenAI’s ‘ChatGPT Health’, realised in January 2026, which allows patients to upload medical records and connects to health apps such as MyFitnessPal for personalised health information
- the Queensland Adult Deterioration Detection System, which uses a scoring system based on patient vital signs to identify patients at risk of clinical decline and predict deterioration up to eight hours before it occurs
- DeepSUVR, an AI model designed to improve the accuracy and consistency of PET scans to assist in the early detection of Alzheimer’s
- GuideStream, a tool that allows practitioners to ask specific, relevant questions about clinical guidelines and receive information in a clear, useful format,2 and
- the Oncology Search Clinician Assistant Registry (OSCAR), designed to search medical literature and clinical trial registries and automatically identify potential targeted therapies for a child’s specific tumour genetics.
The Australian regulatory landscape
The current regulation of AI in healthcare settings within Australia is achieved through a combination of state, territory and national legislation aimed at addressing the exchange of personal information, the regulation of medical devices, and practitioner conduct.
Regulation of practitioners
The Australian Health Practitioner Regulation Agency (AHPRA) has taken a proactive approach to AI regulation by publishing guidance in August 2024 to assist practitioners in meeting their professional obligations when using AI in healthcare. AHPRA’s framework is built on the following core principles:
- Accountability: practitioners must apply human judgement to AI outputs.
- Understanding: practitioners must have a sufficient understanding of the AI tools they use, including their training, intended use, and limitations.
- Transparency: practitioners must inform patients about their use of AI.
- Informed consent: practitioners must involve patients in decisions to use AI tools that require personal data.
Additional guidance for practitioners
Professional associations and other bodies have also produced guidance for practitioners. Of note, the Australian Department of Health, Disability, and Ageing has published an ‘AI Clinical Use Guide’ (the Guide) to support clinicians and their patients in using AI safely and responsibly in patient care.
The Guide urges clinicians to understand how AI technology impacts care delivery and patient outcomes before use. Clinicians must use AI safely and responsibly by applying professional judgement and exercising due care in the context of each patient.
Further, the clinician ultimately remains responsible for confirming the accuracy and completeness of records, results, or documentation that will be acted on by the clinician, other clinicians, or patients.
Practical guidance offered to practitioners includes:
- critically assessing each AI tool, including the scope, use, available evidence and the common limitations and risks of the AI tool
- establishing AI transparency and informed consent procedures, including clear processes for obtaining informed consent from patients for the use of the AI tool. This requires practitioners to have an appropriate understanding of the use and limitations of the AI tool and to continually monitor the functional scope of the tool, which can change or ‘creep’ over time through software updates or upgrades, and
- ensuring that records or results created with AI tools meet the same quality standards as those created manually and align with professional standards by undertaking regular review of AI outputs.
Regulation of devices
The Therapeutic Goods Administration (TGA) regulates the use of AI in therapeutic goods and medical devices. AI tools, including chatbots, large language models, digital scribes, and clinical decision support tools, fall under the Therapeutic Goods Act 1989 (Cth) if their explicit or marketed purpose is to influence or support clinical decisions.3 To be legally supplied in Australia, the AI medical device must undergo a pre-market assessment and be included in the Australian Register of Therapeutic Goods (ARTG) unless excluded or exempt.
The TGA uses a four-tier risk-based classification system that considers the device’s intended purpose and level of harm it may pose. More than 153 AI-enabled devices have been published on the ARTG.4
Examples of some devices and their risk classification include:
- the Ibex platform, developed by Ibex Medical Analytics Pty Ltd, which is intended to assess and interpret pathology slides to identify and/or formulate additional clinical information that may then be used to guide patient management (Class III – High Risk)
- the Merlin.Net Patient Care Network, developed by Abbot Medical, which allows practitioners to remotely monitor and individually program implanted heart devices with AI undertaking assessment of the recorded data (Class III – High Risk), and
- RayStation, developed by RaySearch Laboratories, a software system for radiation therapy and medical oncology that proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorised intended users, RayStation may also be used to administer treatments (Class IIB – Medium to High Risk).
Notably, AI scribes are only regulated by the TGA if their functions go beyond basic transcription to include clinical analysis, diagnosis, or treatment recommendations.5
International developments
As AI tools continue to develop at a rapid pace, regulators will need to keep pace with these emerging technologies to ensure appropriate use and patient safety. International developments provide a glimpse into potential changes to the legal and regulatory landscape in Australia and issues being faced globally.
United States
The United States regulates AI in healthcare through the Food and Drug Administration (FDA). The FDA classifies Software as a Medical Device (SaMD) using a risk-based framework that considers the healthcare situation (critical, serious, or non-serious) and the healthcare decision state (inform, drive, or diagnose).6
There have been examples of technology companies coming under scrutiny in the US as a result of AI-assisted technology. Tech company Character.AI became the subject of a lawsuit by the State of Pennsylvania in May 2026, after a state investigator posing as a patient was told by an AI chatbot that it was licensed to practise medicine in Pennsylvania and the UK and provided a false Pennsylvania medical licence number.7 Similarly, Sharp HealthCare in San Diego is also facing a class-action suit on the basis that its use of AI to “secretly” record patient-clinician conversations violates state consumer privacy laws.
The Plaintiff, Mr Jose Saucedo, claims that Sharp HealthCare allowed clinicians to use an AI-powered ambient documentation tool that ‘surreptitiously recorded’ entire medical consultations using electronic recording devices and cloud-based processing systems without notice or consent.
Together, these cases highlight the vastness of legal and regulatory issues facing the healthcare sector arising from the dynamic nature of AI models.
England
In June 2025, NHS England issued guidance that all AI scribes with summarisation functionality must obtain approval from the Medicines and Healthcare products Regulatory Agency (MHRA).8 This means all AI scribes that have a summarisation function must undergo an extensive governance compliance process in the UK and be registered with the MHRA. This approach offers stronger protections to clinicians and patients than in Australia, as it allows for summarisation alone to be sufficient grounds for the AI tool to be classified as a medical device.
The Medical Protection Society, which represents practitioners accused of wrongdoing has expressed concern that practitioners will be found liable for mistakes made by AI in the treatment of patients and has suggested that AI manufacturers should bear a share of any liability. It has been widely reported that the NHS is facing its first negligence claims involving the use of AI and these cases should be watched closely as they may impact regulation of AI in the Australian healthcare setting.
European Union
The European Union has introduced the Artificial Intelligence Act (AI Act) which sets out specific obligations for providers and deployers of AI systems, including transparency and AI governance requirements. However, liability in relation to AI medical devices is governed by the Product Liability Directive (PLD).
The PLD has undergone a number of changes related to AI, with key changes coming into effect in December 2026. Relevant for practitioners there now appears to be a significantly expanded scope for AI tools to be found ‘defective’ and therefore trigger manufacturer liability. This includes circumstances where the AI tool has breached a separate regulatory requirement, has caused harm and when the AI tool has ‘learned’ since being released and now operates in a way potentially different to how the manufacturer initially intended. These changes suggest that there will likely be significant claims against both practitioners and manufacturers with liability needing to be balanced by the courts.
Key legal risks and challenges
The duty of care
The rapid development of AI poses questions in determining liability in the event of errors or adverse outcomes. AHPRA has confirmed that regardless of what technology is used in providing healthcare, the practitioner remains responsible for delivering safe and quality care, and that TGA approval of a tool does not change a practitioner’s responsibility to apply human oversight and judgement while utilising AI.
However, advancements in technology will lead to the emergence of autonomous AI systems that can assess patients and make recommendations for treatment without practitioner input. Practitioners may soon face liability not just for misusing AI, but for failing to use it.9 Scholars in the field have suggested that once a tool is demonstrably superior to human capability, the law may require the use of that tool in appropriate professional practice.
Accordingly, if an AI model becomes generally accepted as reliable for specified applications, its integration may come to reflect ordinary competent practice, and practitioners who have not adopted that tool may be considered to be acting below the appropriate professional standard.
Record keeping
Health practitioners are obliged to keep and maintain health records that are relevant, up-to-date, and complete in accordance with National Board and each profession’s Code of Conduct.10 The Digital Rights Watch estimates that more than 40% of Australian health practitioners are now using AI scribes.11
While AI scribes reduce administrative burden, they are not without risk. These risks include, but are not limited to:
- the tendency of clinicians to overly rely on AI-generated documentation, transcriptions, or diagnostic suggestions without sufficient verification, and
- AI scribes that generate completely fabricated, unfaithful, or incorrect information, such as inventing diagnoses, prescribing wrong dosages, or recording examinations that never happened.
We have already seen the impact of AI scribes in Australia. For example, a well-publicised recent news story concerning a urologist who was required to apologise to his patient after an AI scribe utilised during their consultation hallucinated an error into her medical records, stating that the patient had taken psychedelic mushrooms when she had not.12
Conversely, regulators are pushing the adoption of AI in healthcare settings. In Health Care Complaints Commission v Della Bruna [2026] NSWCATOD 59, the Tribunal imposed conditions on a medical practitioner to complete education on AI scribe programmes following inadequate paper record-keeping. The Tribunal explained that utilising an AI scribe would enable the practitioner to maintain face-to-face contact with patients, whilst the AI scribe would assist in making timely, comprehensive, and legible records of each consultation.13
As set out above, AI scribes are typically not regulated by the TGA. However, we anticipate that this position will likely change as the technology improves and AI scribes perform more functions using the information generated, such as creating treatment plans and sending correspondence.
Patient-generated data
With emerging technologies like ChatGPT Health and wearable devices becoming increasingly prevalent across the sector, practitioners are anticipated to face major hurdles with patient-generated health data (PGHD), including uncertain data accuracy, heavy workflow burdens, and poor integration into medical records.14
There are many questions that come with the rise of PGHD. These include the extent to which a practitioner should rely on the PGHD, whether a practitioner is required to respond to PGHD, whether a contributory negligence argument may arise against a patient or device manufacturer for inaccurate PGHD, and how PGHD ought to be recorded in the practitioners’ medical records.
While there is no clear guidance in Australia, the US-based Society for Participatory Medicine sets out strategies for managing risk. These strategies include:
- working with patients to achieve a common understanding of the types of PGHD patients will share, how the sharing will take place, and which members of the clinical team will review the information and how often
- putting a medical emergency protocol in place with the patient, and
- using appropriate judgement in deciding whether PGHD should be included in the clinical record.
Practitioners remain responsible for exercising their professional judgement when relying on AI and PGHD to diagnose, treat, and care for their patients.
Conclusion
AI is transforming healthcare, but its integration presents complex legal and regulatory challenges. Practitioners must navigate evolving frameworks, address liability and ethical concerns, and ensure compliance with professional standards. Practitioners must also stay abreast of developments to mitigate risks and support responsible AI adoption, while insurers must understand the emerging risks that these practitioners face.
For now, practitioners remain responsible for the AI tools used in their practice, and care must be taken to ensure their safe and appropriate use to minimise the risk of claims.
Footnotes
4. AI-enabled medical devices in the ARTG | Therapeutic Goods Administration (TGA)
5. Digital scribes | Therapeutic Goods Administration (TGA)
6. Global Approach to Software as a Medical Device | FDA
7. AI: “Deepfake doctor” chatbot is hit with lawsuit in US | The BMJ
10. Off The Record - AI Scribes in Australian Healthcare
12. Error by AI scribe during medical appointment leaves patient devastated - ABC News
13. Health Care Complaints Commission v Della Bruna [2026] NSWCATOD 59
14. Information Quality Challenges of Patient-Generated Data in Clinical Practice - PMC
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
