Last week, in our first session of Advanced Legal Research at the University of Wisconsin Law school, we discussed ChatGPT as a tool for conducting legal research. Why take the time to learn and conduct legal research if an AI can do it for you? Our students raised a lot of great points and showed a sophisticated understanding of the pros and cons of the tool. Overall, they were pretty skeptical.
We started with a demo by asking ChatGPT the elements of a claim for age discrimination (response was very good), then built on that by narrowing it to Wisconsin law (also good). Next, we asked for the leading cases (iffy), some defenses (pretty good), and finally asked it to draft a complaint to the Wisconsin Equal Rights Division. The complaint that it drafted was for termination of employment and our issue was for failure to hire, so we asked it to revise the complaint accordingly and it did.
Then we broke into groups and asked students to discuss their impressions of ChatGPT and its use and limitations in legal practice. Here are some of the issues that were raised:
- Chat GPT instills a false sense of confidence in its responses (some of which are flat-out wrong), despite disclaimers that you should contact an attorney (which will probably be ignored)
- Students were concerned that the source of the data powering the AI is unknown. Is it coming from general sources? legal sources? Who is the author of these sources? How current is the information?
- ChatGPT often doesn’t provide citations to the answers it provides, although you can ask it to do so with a limited degree of success
- Students were concerned that providing information and factual details might breach client confidentiality
- Students were hesitant to stake their professional reputations on the quality of answers generated by an AI – which led to:
- Chat GPT may be a good starting point but is not where you should end your research
- There is value in taking time to learn the law and process of legal research so that you can evaluate the accuracy of responses and provide further analysis
- ChatGPT could be helpful for those who can’t afford an attorney, but students were concerned about pro se overreliance
- Finally, and this is the major theme of our whole Advanced Legal Research course: It’s important to be a smart consumer – to understand the benefits and limitations associated with this and any other research tool so that you use it wisely and appropriately