\n
\n
![\"Enhancing](\"https:\/\/image.blockchain.news:443\/features\/D8E08E86F8EDBDDCD68414CF49BDD8B1401B11A69515DFF98E6B2B03EE9CF9D7.jpg\")
\n
\n <\/a>
\n <\/figure>\n
Developers seeking to deploy large language model (LLM) applications more safely and quickly now have a robust solution with LangChain Templates and NVIDIA NeMo Guardrails, according to NVIDIA Technical Blog.<\/p>\n
Benefits of Integrating NeMo Guardrails with LangChain Templates<\/h2>\n
LangChain Templates offer developers a new way to create, share, maintain, download, and customize LLM-based agents and chains. These templates enable the swift creation of production-ready applications, leveraging FastAPI for seamless API development in Python. NVIDIA NeMo Guardrails can be integrated into these templates to provide content moderation, enhanced security, and evaluation of LLM responses.<\/p>\n
As generative AI continues to evolve, integrating guardrails ensures that LLMs used in enterprise applications remain accurate, secure, and contextually relevant. The NeMo Guardrails platform provides programmable rules and runtime integration to control user inputs before engaging with the LLM and to validate the final LLM output.<\/p>\n
Setting Up the Use Case<\/h2>\n
To demonstrate the integration, the blog post explores a Retrieval-Augmented Generation (RAG) use case using an existing LangChain template. The process involves downloading the template, modifying it to suit the specific use case, and then deploying the application with added guardrails to ensure security and accuracy.<\/p>\n
LLM guardrails help minimize hallucinations and keep data secure by implementing input and output self-check rails that mask sensitive data or rephrase user inputs. For example, dialog rails can influence how LLMs respond, and retrieval rails can mask sensitive data in RAG applications.<\/p>\n
Downloading and Customizing the LangChain Template<\/h2>\n
To begin, developers need to install the LangChain CLI and the LangChain NVIDIA AI Foundation Endpoints package. The template can be downloaded and customized by creating a new application project:<\/p>\n
pip install -U langchain-cli
\npip install -U langchain_nvidia_aiplay
\nlangchain app nvidia_rag_guardrails --package nvidia-rag-canonical<\/code><\/pre>\nThe downloaded template sets up an ingestion pipeline into a Milvus vector database. In this example, the dataset contains sensitive information regarding Social Security Benefits, making guardrail integration crucial for secure responses.<\/p>\n
Integrating NeMo Guardrails<\/h2>\n
To integrate NeMo Guardrails, developers need to create a directory named guardrails<\/strong> and configure the necessary files such as config.yml<\/code>, disallowed.co<\/code>, general.co<\/code>, and prompts.yml<\/code>. These configurations define the guardrail flows that control the chatbot’s behavior and ensure it adheres to predefined rules.<\/p>\nFor example, a disallowed flow might prevent the chatbot from responding to misinformation, while a general flow might define acceptable topics. Self-checks for user inputs and LLM outputs are also implemented to prevent cybersecurity attacks like prompt injection.<\/p>\n
Activating and Using the Template<\/h2>\n
To activate the guardrails, developers need to include the configurations in the config.yml<\/code> file and set up the server for API access. The following code snippets show how to integrate the guardrails and set up the server:<\/p>\nfrom nvidia_guardrails_with_RAG import chain_with_guardrails as nvidia_guardrails_with_RAG_chain
\nadd_routes(app, nvidia_guardrails_with_RAG_chain, path=\"\/nvidia-guardrails-with-RAG\")<\/code><\/pre>\nfrom nvidia_guardrails_with_RAG import ingest as nvidia_guardrails_ingest
\nadd_routes(app, nvidia_guardrails_ingest, path=\"\/nvidia-rag-ingest\")<\/code><\/pre>\nDevelopers can then spin up the LangServe instance with the command:<\/p>\n
langchain serve<\/code><\/pre>\nAn example of a secure LLM interaction might look like this:<\/p>\n
\"Question\": \"How many Americans receive Social Security Benefits?\"
\n\"Answer\": \"According to the Social Security Administration, about 65 million Americans receive Social Security benefits.\"<\/code><\/pre>\nConclusion<\/h2>\n
This integration of NeMo Guardrails with LangChain Templates demonstrates a robust approach to creating safer LLM applications. By adding security measures and ensuring accurate responses, developers can build trustworthy and secure AI applications.<\/p>\n