A naukri.com initiative
Siliconangle
1M
366
Image Credit: Siliconangle
Data in the generative AI era: We need more knowledge engineers
- The concerns over data privacy and cyber issues, employee decisions based on erroneous information, and employee misuse and ethical risks have highlighted the need for a professional who understands the data's context to point organizations to the right information.
- AI projects advance from the proof of concept to production, so organizations have to pay serious attention to the data that is used for training and inference.
- AI has produced rightsized models based on a variant of the 80/20 rules by having just enough data and model to deliver results that are considered good enough.
- Understanding the source and knowing how and where the metadata flows are essential steps to understanding the pulse of information across an organization.
- Context engineering is emerging as a discipline that provides a systematic solution to capture context and make it explicit while knowledge engineering update the reference librarian with software engineering skills and borrows from semantic disciplines such as ontology development, knowledge representation and related skills for representing it with graphs.
- Demand for knowledge engineers has coincided with the rising prominence of knowledge graphs to get applied to AI so that training and running models do not generate millions of dollar cloud computing bills and that AI models stay appropriately grounded.
- While AI can assist in the legwork of harvesting data, ultimately, it requires the skills of knowledge engineers to make the final call.
- Knowledge engineering has roots in library science, so it may be time to update the old Harvard Business Review article naming the present-day equivalent of the reference librarian as the sexiest job of the 21st century.
Read Full Article
22 Likes
For uninterrupted reading, download the app