AWS is placing its bets on the idea that by making its Automated Reasoning Checks feature on Bedrock widely available, more enterprises and regulated industries will gain the confidence to utilize and deploy a greater number of AI applications and agents. 

The company is also hopeful that by introducing methods like automated reasoning—which employs math-based validation to establish ground truth—businesses will be more inclined to embrace the realm of neurosymbolic AI. AWS believes this will mark the next significant advancement and serve as its most distinct differentiator in the AI landscape. 

Automated Reasoning Checks allow enterprise users to confirm the accuracy of responses and identify model hallucinations. AWS introduced Automated Reasoning Checks on Bedrock during its annual re: Invent conference in December, asserting it can detect nearly 100% of all hallucinations. Initially, a limited group of users could access the feature through Amazon Bedrock Guardrails, where organizations are able to establish responsible AI policies.

Byron Cook, a distinguished scientist and vice president at AWS’s Automated Reasoning Group, conveyed in an interview with VentureBeat that the preview rollout demonstrated that systems like this are effective in an enterprise setting. It also aids organizations in appreciating the value of AI that can integrate symbolic or structured thinking with the neural network nature of generative AI. 

AI Scaling Hits Its Limits

Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:

• Turning energy into a strategic advantage

• Architecting efficient inference for real throughput gains

• Unlocking competitive ROI with sustainable AI systems

Secure your spot to stay ahead: https://bit.ly/4mwGngO

“There’s this concept of neurosymbolic AI, which is the category under which automated reasoning falls,” Cook explained. “The growing interest in neurosymbolic AI made users, while engaging with the tool, realize the significance of this work.”

Cook noted that some clients allowed AWS to examine their data and the documents used to assess the answers as correct or incorrect. They discovered that the work produced by the tool performed comparably to humans equipped with a rule book. He further mentioned that the notion of truth or correctness can often be open to interpretation, a challenge that automated reasoning mitigates. 

“It was truly astonishing! It was incredible to see individuals with backgrounds in logic engaging in internal discussions about what is true or not, and within just a few messages, they would point to the tool and recognize that it is indeed correct,” he remarked. 

AWS has introduced new features to Automated Reasoning Checks for its general release. These features include:

• Support for adding large documents of up to 80k tokens or up to 100 pages 

• Simplified policy validation by saving validation tests for repeated executions

• Automated scenario generation based on pre-saved definitions

• Natural language suggestions for policy feedback

• Customizable validation settings

Cook stated that Automated Reasoning Checks validate truth or correctness in an AI system by ensuring that a model did not hallucinate a solution or response. This capability could provide regulators and regulated enterprises concerned about the non-deterministic nature of generative AI returning incorrect responses with greater assurance. 

Neurosymbolic AI and Proving Truth

Cook highlighted the idea that Automated Reasoning Checks help demonstrate many of the principles of neurosymbolic AI. 

Neurosymbolic AI refers to the fusion of neural networks, which language models utilize, with the structured thinking and logic of symbolic AI. While neural networks identify patterns from data, symbolic AI relies on explicit rules and logical problems. Foundation models often depend on neural networks or deep learning, but since their responses are pattern-based, they are susceptible to hallucinations, a concern that continues to trouble enterprises. However, symbolic AI lacks flexibility without manual instructions.

Prominent figures in AI, such as Gary Marcus, have emphasized that neurosymbolic AI is vital for achieving artificial general intelligence. 

Cook and AWS are enthusiastic about introducing neurosymbolic AI concepts to enterprises. VentureBeat’s Matt Marshall discussed AWS’s emphasis on methods like automated reasoning checks and the combination of math and logic with generative AI to reduce hallucinations in a podcast. 

Currently, few companies offer productized neurosymbolic AI solutions. These include Kognitos, Franz Inc, and UMNAI.

Bringing Math to Validation

Automated reasoning operates by applying mathematical proofs to models in response to a query. 

It uses a method known as satisfiability modulo theories, where symbols have predefined meanings, and it addresses problems involving both logic (if, then, and, or) and mathematics. Automated reasoning employs this method to evaluate a model's responses and checks them against a set of policy or ground truth data without necessitating multiple tests of the answer. 

For instance, in an enterprise setting, if they want to verify the accuracy of a financial audit, the model might respond that a report contains unapproved payments. Automated reasoning checks break this down into a logical string:

(forall ((r Report))

  (=> (containsUnapprovedVendorPayments r)

      (shouldEscalate r)))

It then examines the definitions, variables, and types set by the user on Bedrock Guardrails and resolves the equation to confirm that the model responded accurately and based on truth.

Making Agents Provably Correct

Cook mentioned that agentic use cases could significantly benefit from automated reasoning checks, and by providing greater access to the feature through Bedrock, its utility can be demonstrated. However, he cautioned that automated reasoning, along with other neurosymbolic AI techniques, is still in its nascent stages. 

“I believe it will impact agentic AI, though, of course, the agentic work is quite speculative at the moment,” Cook said. “There are several techniques like this that involve discovering ambiguity in a statement, identifying the key differences between possible interpretations, and then returning to you for refinement. I think this will be crucial in terms of the emotional journey I observed customers experiencing when they began experimenting with generative AI a couple of years ago.”