Copilot AI as an editorial assistant: meh
A few days ago Microsoft Copilot showed up in Word. It offers to rewrite stuff for me. So I tried it out on a few paragraphs. I have to admit it’s not blowing me away.
Let’s have Copilot rewrite that paragraph:
Recently, Microsoft Copilot has been integrated into Word. It provides an option to rewrite text. After experimenting with it on several paragraphs, I must acknowledge that its performance has not met my expectations.
The original is better, no?
Here’s a paragraph from a piece I’m working on:
Liberals and neoliberals disagree about the role of the Supreme Court and the proper approach to constitutional interpretation. Very roughly, liberals tend to support a flexible approach to constitutional interpretation that allows the activities of government and the rights of citizens to adjust in response to major changes in economic and social life, such as the industrial revolution and changing attitudes towards historically disfavored groups. The liberal approach to constitutional interpretation goes under names like “living constitutionalism” or “common law constitutionalism”, which emphasize the possibility of informal constitutional change. Liberals do not believe that the constitution is infinitely flexible. The words in the constitution are sometimes clear and binding. However, liberals believe that our understanding of the meaning of the constitution should be guided by broad purposes and values that allow adjustment to new circumstances over time.
Here’s the Copilot rewrite:
Liberals and neoliberals differ on the Supreme Court’s role and constitutional interpretation. Liberals favor a flexible approach, allowing government activities and citizens’ rights to adapt to significant economic and social changes like the industrial revolution and evolving views on marginalized groups. Called “living constitutionalism” or “common law constitutionalism,” this approach suggests informal constitutional change. While not infinitely flexible, liberals think the constitution’s meaning should be guided by broad purposes and values that adjust over time.
Shorter is always better, of course, but overall I think the Copilot version is worse.
Here’s the next paragraph from my piece:
Many neoliberal legal theorists reject this liberal view of constitutional interpretation. They argue that the meaning of the constitution was fixed at the time of its ratification, that the original meaning of the constitution is inconsistent with many of the tax, spending, and regulatory activities characteristic of modern government, and that the supreme court should restrict the size and scope of government by interpreting the constitution in a manner consistent with the understanding of those who framed or ratified it.
Here’s the Copilot rewrite:
Many neoliberal legal theorists disagree with the liberal approach to constitutional interpretation. They believe the constitution’s meaning was set at its ratification and that this original meaning opposes many modern governmental tax, spending, and regulatory practices. They argue the supreme court should limit government size and scope by interpreting the constitution based on the framers’ and ratifiers’ understanding.
I suppose it’s useful for Copilot to suggest wording changes like “disagree” for “reject”, although I don’t think that’s an improvement in this case. Changing “fixed” to “set” is a problem: originalists are committed to what they explicitly call “the fixation thesis”, so using “fixed” makes more sense than using “set”. The second sentence is off – it seems to attribute agency to “original meaning”. That’s weird. The third sentence is also worse than the original. “the supreme court should restrict the size and scope of government” is better than “the supreme court should limit government size and scope”, to my ear. In addition, originalists disagree about whether we should care about the original understanding of framers or ratifiers or the public. Changing “or” to “and” is a mistake.
I do not doubt that AI can help people write, and its value will grow over time. But given my particular needs it just hasn’t been very helpful so far . . .
I must admit that I have never accepted the interference of AI in my professional nor personal life. My professional writing was articulate, technical, and dry with sentences that conveyed nuanced meaning via transitional connectors sometimes resulting in longer sentences than considered generally appropriate in formal writing yet useful when discussing complex technical issues which would be associated with high costs (e.g., like that). My professional writing days ended almost ten years ago. My personal writing has always been stylized juxtaposing academic technical perspectives with sociological and psychological undertones augmented by sharply ironic and cynical candor. AI is rules based clear, concise, and boring mediocrity taken to the extreme. IOW, AI can go suck it.
I’ve done a lot of line editing (grant proposals and scientific manuscripts) for people who don’t speak English as their first language. This could probably help in many cases. The risk is when the effort to improve the English actually changes the sense.
@Joel,
AI is far more artificial than it is intelligent. I.e., AI is purpose built fundamentally as a expert system with heuristic intelligence sufficient to acquire broader knowledge base from experience upon which it can improve itself. This is not general intelligence which has never been programmed into any machine and is still exceedingly rare in human beings. The Turing Test is plain wrong. Alan’s Pilot ACE computer in 1950 was not far enough down the technology yellow brick road for AT to have developed a critical view of his own vision. Besides his interpersonal skills were more like a robot’s than a human just for starters.
Build it and they will come, but pay for it and they will build it. AI Catch-22.
Good read at link below on the world fastest supercomputers which are all still purpose built research machines rather than general intelligence AI (Asimov’s dream as well as Turing). One can easily understand how human intelligence could be underestimated given the historical evidence.
https://www.livescience.com/technology/computing/top-most-powerful-supercomputers
“…The risk is when the effort to improve the English actually changes the sense…”
[ Yes, how would it know? Per Asimov, AI is just a naive child. Question is would the linguistically challenged author know or would they have faith in AI? ]
@weakely,
The authors for whom I edited read plenty of English articles. So they could read edits of their writing for the science. Their problem was composing English grammar and syntax.
I hope they do not incorporate this faux writing into grade schools thereby supplanting the teaching of writing. Such would be rather dry and boring thereby removing a degree of incorporated style by the author and the intellect of the same.
Still waiting on a call to come and pick up my computer.
Thanks Eric.
Maybe it should have a “dead band” logic where it looks at the original material and “scores” it and if it “scores” above a certain level just tell the user that it’s good as is. Many control systems have logic like ‘demand just was input at 95, but the system is 94 so let’s ignore the demand for now’.