How AI Can Create Actual Value

Predictions on how generative AI will transform the legal profession abound. Uncertainty around this emerging technology is also a hot topic, especially for lawyers, whose training makes them (rightly!) skeptical of whether AI tools are trustworthy. Professionals across our industry are trying to figure out how law firms can actually use AI effectively and safely.

A recent study examining how individuals can create — and destroy — value with generative AI provides some additional data to consider as organizations adopt AI tools. The study showed that when used the right way for the right tasks — such as basic creative ideation or content creation — generative AI improved individuals’ performance. That’s because the tools have a lot of data to draw on to suggest ideas and text for certain genres of content that are highly standardized in form and function.

But when used for more specialized and complex tasks — like combing through performance data and executive interviews to determine the cause of a company’s challenges — individuals who used generative AI fared worse than those who relied on their own analytical skills. Humans are good at evaluating nuanced qualitative and quantitative data to answer complex questions. Generative AI is not (yet) sophisticated enough to do that work.

What’s tricky, the study emphasized, is that people don’t always grasp which types of tasks generative AI can help with. The study found that individuals tend to mistrust the technology for tasks where it can add value and trust it too much for tasks where it isn’t helpful. In other words, we are clinging to our low-level tasks and trying to outsource the more complex ones, when we should be doing the opposite.

How generative AI can support marketers and business development professionals

An encouraging sign is that marketers and business development professionals are eager to try this new technology, even if their firms aren’t quite there yet. As a starting point, legal marketers and business development professionals need to understand when generative AI is up to specific task.

Here’s where we’re already seeing success using generative AI:

• Writing. Marketers are using generative AI platforms to boost their productivity with writing. Tools like ChatGPT and Content At Scale may be helpful for writing first drafts of press releases, bios, blog content and social media posts, award nominations and speeches. Common sense dictates, however, that the more specialized and/or complex the set of criteria you are trying to meet (say, for a Chambers submission), the less likely a one-size-fits-all piece of content is going to work for you. Getting satisfying output from your AI tool depends on the quality of the prompt you give it to generate the content. And, of course, you must carefully review your AI-generated content for factual errors and potential plagiarism. Treat anything generated by the tool as a first draft that needs careful editing. And never feed confidential information to an AI tool.
• Research and SEO. Marketers are using AI for tasks involving research and SEO. For example, you can use AI tools to identify the most popular hashtags for a practice area to use in social media posts or the most searched-for terms in a practice area. In the context of legal marketing, this is a pretty minor contribution. But AI tools may be able to take small, repetitive tasks like this off your plate.

A few more risks to monitor

Another interesting statistic from the study sheds light on how reliance on AI may narrow a slate of solutions. The technology’s “relatively uniform output” was shown to reduce a group’s diversity of thought by 41%. We know from a large body of research on human diversity in group problem-solving that when people from different backgrounds work together, they come up with more creative solutions than homogenous groups. If everyone in that group defers to the answer provided by the AI tool, however, they may lose that innovative thinking. Firms will need to do take action to counter this phenomenon.

To further complicate things, the study also found that individuals often failed to question whether the answer the AI tool provided to a complex business problem was actually correct. Many users “accepted the tool’s erroneous output at face value,” making it clear that we have a long way to go in educating users about the appropriate level of skepticism needed when engaging with generative AI. As we said above, for today’s more complex business questions, human analytical skills are still superior to what AI can generate.

For now, legal marketers will be best served by using generative AI tools to increase their efficiency with low-level tasks that do not require much complex thinking. Basic content and research projects are two safe areas to experiment. Going forward, as the tools become more sophisticated, firms will need to carefully consider how to train law firm staff and lawyers to expand into more complex use cases without sacrificing quality or innovation.