❯What does AI need to be really effetive?
Perspectives on the origin of the gaps between human production and AI output.
If a lion could speak, we could not understand him—Ludwig Wittgenstein
The vision of the future in the 1920's included flying cars with gyroscope stabilization, and collapsible blades that allowed the vehicle to transition between road travel and aerial flight. These machines would be constructed from lightweight, fireproof materials and potentially powered by wireless electricity. These ideas have come and gone like meteors, what has remained is the pattern of waves of hype and promises of an almost mytical future of ease and fortune that, looking back at history, it has never realized itself.
This type of overconfidence about the future seems to be a feature, not a bug, so there's nothing new here. Still, there is a complex relationship, one I can't quite put my finger on, between our capacity to create abstractions and fall infatuated with our own mental creations, versus our legitimate search for knowledge and improvement.
Epistemic gaps
Our mental models are not necessarily the truth nor true representations of reality. There's a gap between the two and some people are more receptive to this difference than others.
The AI-positivists propose the idea that AI produces outputs that can, in sufficiently advanced cases, be treated as epistemically sufficient and not requiring significant human interpretation to have practical value. The most convinced supporters would even conclude that subjective interpretation, theory, or human judgment becomes secondary.
The physicalist interpretation would lead us to the sober understanding of AI as a system generating token sequences through statistical optimization. The Output does not by itself constitute knowledge or truth, which requires human interpretation and validation. The accuracy of the output is a property of the optimization process, not an intrinsic quality of the system.
The point of disagreement can be summarized as follows: AI-positivism (as defined above) sees generated outputs as epistemically sufficient, while physicalism would see those outputs as dependent on human interpretation in order for them to have epistemic content.
Where do you stand on this subject?
Perhaps there's a third position: AI produces outputs that is not itself knowledge, but can be used, with sufficient reliability and in a variety of cases, as input into knowledge production when humans add validation and interpretation. I would tend to align with this position and, as a consequence, that AI does not replace humans from an epistemic perspective.
You might say that AI does partially replace humans from a functional or economical perspective. In fact AI can generate at least a decent webpage, even with some realistic 3D elements. But I would contend that the epistemic gap spills over into the functional and economical realms: even if the webpage is of sufficient quality, the result is just the tip of the iceberg. In other words, there's much more to a well executed project than the code itself that cannot be reduced to an interface, a button, low latency measurements and other benchmarks, and AI can't follow you there.
Micro productivity, Macro Mess
While coding assistants can make (some) specific tasks easier, that sliver of work lives within the context of a complex organization. The complexity is such that the increase in productivity is largely bottlenecked by managerial funnels, by hard project constraints such as compliance, or technicalities that require human attention such as security, cloud infrastructure or software architecture.
Solving problems we cannot define
To put it simply, AI is most effective when used to solve problems we're already quite good at defining. When goals are ambiguous and definitions are vague neither AI nor humans perform well. It's precisely the difficult problems we want AI to solve that we can't define sufficiently well, nor we can define what the solution to those problems would look like. In the end it's humans determining what problems are worth solving and what "success" looks like.
Conclusion
Integrating AI effectively is still difficult. Interacting with AI is still quite dull, Its output feels dry and ineffective when put in complex human contexts which are rich of interpretations and interactions. The difficulty to capture exactly what's missing is probably due to the fact that it cannot be reduced to easily measurable quantities, such as accuracy, natural language fluency, retrieval accuracy and other benchmark scores. The missing dimensions are more like human-like: situational awareness, shared meaning, cultural presuppositions, contextual grounding and more.