Efforts to develop Artificial Intelligence (AI) capabilities to create software recently generated headlines when stories were published describing how OpenAI built a coding tool named ChatGPT by generating AI rules from scanning millions of lines of source code housed in repositories within GitHub. GitHub was purchased by Microsoft in 2018 and Microsoft not only allowed the scanning of these GitHub repositories for this purpose, it invested in OpenAI to fund the effort. The main concern raised with that news was that such a capability would essentially allow the creation of new, derivative works using patterns -- if not complete actual snippets of source -- from other parties' copyrighted code without prior permission or even compensation.
OpenAI is now attempting to augment the training process for its product with an additional approach summarized here on the web site Semafor:
OpenAI's new idea? Hire over one thousand developers to solve a variety of "canned" programming problems, have those developers create a prose description of how they defined the problem and structured their code to solve it, then feed the code and the prose into databases to be scanned by training algorithms for ChatGPT. The idea behind this approach is to create a better mapping BACKWARD from working code examples to higher level non-technical "asks" that might come from "business owners" to further streamline the process and make development faster and less expensive. Extrapolated to an extreme, a result of this approach could be to allow a non-technical "business owner" to type in a description of a capability they want in their lingo and map it to actual code / data concepts required to implement and build it.
At first glance, this idea of enhancing AI training by having it look further up the development cycle closer to business users makes sense. It might even help avoid legal disputes over copyright violations stemming from focusing solely on source patterns, or make it easier to deflect copyright claims in court.
Except...
The effort involves having the DEVELOPER solve the problem then provide a PROSE explanation on how they approached the problem then feeding (code + developer prose) into the AI. Can anyone spot the problems with that?
First, the prose being fed to train the AI comes from the DEVELOPER, not the original "business owner." That's not how most large-scale software systems are created. Developers are the LAST in a line of specialties involved in parsing someone's "requirements" prose from one vernacular (perhaps the "business owner's" requirements writers) to another (perhaps the developer team's business analyst team) to another (the enterprise architects) to another (the solution team's architects) then the actual developers. Every one of those handoffs requires re-interpretation and introduces new semantics which can obscure or distort intent.
The second problem is that not all developers excel at explaining in prose terms WHAT they are building, WHY they are building it that way and HOW it will actually work. The code may be perfect but there is no guarantee someone outside the head of the developer will be able to understand the solution from the developer's description of it. Granted, some of the best developers have TREMENDOUS communication and coding skills which makes them such good developers. Unfortunately, not everyone works for Lake Woebegone Systems, where all of the developers are above average in this area. In many cases, the prose created as explanation for what / why will likely be so poorly structured it could prevent an AI from accurately parsing / extrapolating it to new scenarios.
Thirdly, companies building imaginative new systems with Google-level engineering teams likely have better engineers and better requirements writers to use as training fodder but chances are, these upper-crust firms are probably the least interested in using this type of AI. Their teams can create a web service in a few days and launch a new feature on their customer portal in a few weeks. The target market for these AI capabilities is the typical Fortune 500 firm needing ongoing development of tools for internal HR / accounting / analytics systems. Those systems are typically controlled by departmental executives who
- have little or no understanding of their existing or desired business processes
- have little or no understanding of the technical aspects of their existing or desired business processes
- have little or no interest in learning about the details to provide better process designs or requirements
- don't trust the team within the company formally assigned to do development / integration work on their behalf
- often have their own parallel sets of business owners, project managers and business analysts to provide direction to the actual development team
This is a horrible environment in which to work for all parties involved but it is driven by turf battles which inevitably crop up when leaders value control and autonomy (or the illusion of control and autonomy) over efficiency, cost effectiveness and security of the larger firm. It is these layers of overhead for "client" ownership / project management / business analysis added to the "delivery organization" ownership / project management / business analysis that turn $500,000 projects into $2,000,000 dollar projects or $3,000,000 into $15,000,000 projects that are delivered months / years late with only small fractions of the originally promised functionality. Until those duplicate resources are eliminated and the underlying turf battles that created them are eliminated, pointing AI systems at the drivel that passes for "requirements" in Corporate America will barely move the needle on any measure of quality, functionality or cost.
WTH
