Jun 18, 2024
Of all the mental models and metaphors that we’ve cycled through in the past couple of years to conceptualize the development of generative AI tools, the one I keep coming back to is the idea of a “jagged technological frontier.” First advanced in a joint study working paper published last September, the jagged frontier succinctly describes a situation where “some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI.”
To my mind, the mental model of the jagged frontier remains particularly useful because it reminds us that (1) the capabilities of today’s AI tools are NOT uniform across a wide range of tasks, (2) those capabilities – like a frontier – are NOT fixed, and (3) in light of the previous considerations, we should be careful NOT to regard AI as a singular, monolithic thing that can usefully be evaluated on some kind of oversimplified transformative / trash binary.
Not all AI models are generative AI models, and even when we restrict our focus to the latter, our conversations about capability, productivity, and potential benefit from specificity. Shockingly, real-world utility comes down to real-world use cases.
Can a given LLM generate a huge volume of convincingly human-ish text on a given theme? You bet. Can it write jokes? In a generic sense, yes. Can it write a comedy routine that is genuinely and consistently funny? Apparently not.
Ok, but can it write an email? Yes. Can it write an email that will leave the recipient feeling valued and like something other than a box to be checked? Again, perhaps not.
For the examples above, the tasks at hand require not only writing on theme coherently in a specified format (Check!) but also elements of creativity, serendipity, warmth, authenticity, familiarity, etc that gen AI tools don’t – and perhaps shouldn’t – do capably in our stead. Clearly, there are tasks these tools reliably do well, tasks they don’t do so well, and tasks for which they are wildly ill suited. Knowing which tasks are which depends on understanding the current capabilities of the tools AND understanding exactly what it is that constitutes or contributes to “success” on a specific task.
And this is something that all of the people I know who have meaningfully and productively incorporated gen AI tools into their workflows have in common: They’ve spent a significant amount time exploring the jagged frontier through a personal lens – relying on their close knowledge of specific task requirements to do a personal evaluation of tool capabilities.
I’ve tried to emulate this approach myself, and indeed, the best value and productivity punch I’ve found with gen AI tools has involved very specific use cases in my work. I’ll offer two examples with key commonalities and a key distinction between them: As a facilitator, I’ve found Framework Finder – a custom GPT created by Ethan Mollick & co – useful for suggesting relevant tools and frameworks to apply to a range of problems that clients and/or learners might encounter, and I’ve leveraged ChatGPT extensively as a partner for creating (and illustrating) sets of divergent future scenarios.
In both cases, I’m relying on the AI tool to provoke or restructure my thinking rather than doing the thinking (or finished-product writing) for me, and in both cases, I don’t need the AI to be “right” but rather to push me toward paths I might not readily see. And then there’s this distinction: One of these use cases (the scenario work) leverages capabilities that have been available in several generations of LLMs; the other (involving a custom GPT) depends on a narrow-focus tool added to ChatGPT only about a month ago.
This, I suppose, is key to the enduring utility of the “jagged frontier” model: It reminds us that the space is always changing, always being negotiated, always capable of being seen from another vantage point, and always rewarding closer-look curiosity and exploration rather than certainty about is or what can be.
@Jeffrey
