Skip to main content
Open In ColabOpen on GitHub

ChatGroq

This will help you get started with Groq chat models. For detailed documentation of all ChatGroq features and configurations head to the API reference. For a list of all Groq models, visit this link.

Overview​

Integration details​

ClassPackageLocalSerializableJS supportPackage downloadsPackage latest
ChatGroqlangchain-groq❌beta✅PyPI - DownloadsPyPI - Version

Model features​

Tool callingStructured outputJSON modeImage inputAudio inputVideo inputToken-level streamingNative asyncToken usageLogprobs
✅✅✅❌❌❌✅✅✅✅

Setup​

To access Groq models you'll need to create a Groq account, get an API key, and install the langchain-groq integration package.

Credentials​

Head to the Groq console to sign up to Groq and generate an API key. Once you've done this set the GROQ_API_KEY environment variable:

import getpass
import os

if "GROQ_API_KEY" not in os.environ:
    os.environ["GROQ_API_KEY"] = getpass.getpass("Enter your Groq API key: ")

To enable automated tracing of your model calls, set your LangSmith API key:

# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
# os.environ["LANGSMITH_TRACING"] = "true"

Installation​

The LangChain Groq integration lives in the langchain-groq package:

%pip install -qU langchain-groq

Instantiation​

Now we can instantiate our model object and generate chat completions.

Reasoning Format

If you choose to set a reasoning_format, you must ensure that the model you are using supports it. You can find a list of supported models in the Groq documentation.

from langchain_groq import ChatGroq

llm = ChatGroq(
    model="deepseek-r1-distill-llama-70b",
    temperature=0,
    max_tokens=None,
    reasoning_format="parsed",
    timeout=None,
    max_retries=2,
    # other params...
)

Invocation​

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg
AIMessage(content="J'aime la programmation.", additional_kwargs={'reasoning_content': 'Okay, so I need to translate the sentence "I love programming." into French. Let me think about how to approach this. \n\nFirst, I know that "I" in French is "Je." That\'s straightforward. Now, the verb "love" in French is "aime" when referring to oneself. So, "I love" would be "J\'aime." \n\nNext, the word "programming." In French, programming is "la programmation." But wait, in French, when you talk about loving an activity, you often use the definite article. So, it would be "la programmation." \n\nPutting it all together, "I love programming" becomes "J\'aime la programmation." That sounds right. I think that\'s the correct translation. \n\nI should double-check to make sure I\'m not missing anything. Maybe I can think of similar phrases. For example, "I love reading" is "J\'aime lire," but when it\'s a noun, like "I love music," it\'s "J\'aime la musique." So, yes, using "la programmation" makes sense here. \n\nI don\'t think I need to change anything else. The sentence structure in French is Subject-Verb-Object, just like in English, so "J\'aime la programmation" should be correct. \n\nI guess another way to say it could be "J\'adore la programmation," using "adore" instead of "aime," but "aime" is more commonly used in this context. So, sticking with "J\'aime la programmation" is probably the best choice.\n'}, response_metadata={'token_usage': {'completion_tokens': 346, 'prompt_tokens': 23, 'total_tokens': 369, 'completion_time': 1.447541218, 'prompt_time': 0.000983386, 'queue_time': 0.009673684, 'total_time': 1.448524604}, 'model_name': 'deepseek-r1-distill-llama-70b', 'system_fingerprint': 'fp_e98d30d035', 'finish_reason': 'stop', 'logprobs': None}, id='run--5679ae4f-f4e8-4931-bcd5-7304223832c0-0', usage_metadata={'input_tokens': 23, 'output_tokens': 346, 'total_tokens': 369})
print(ai_msg.content)
J'aime la programmation.

Chaining​

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)
API Reference:ChatPromptTemplate
AIMessage(content='The translation of "I love programming" into German is "Ich liebe das Programmieren." \n\n**Step-by-Step Explanation:**\n\n1. **Subject Pronoun:** "I" translates to "Ich."\n2. **Verb Conjugation:** "Love" becomes "liebe" (first person singular of "lieben").\n3. **Gerund Translation:** "Programming" is translated using the infinitive noun "Programmieren."\n4. **Article Usage:** The definite article "das" is included before the infinitive noun for natural phrasing.\n\nThus, the complete and natural translation is:\n\n**Ich liebe das Programmieren.**', additional_kwargs={'reasoning_content': 'Okay, so I need to translate the sentence "I love programming." into German. Hmm, let\'s break this down. \n\nFirst, "I" in German is "Ich." That\'s straightforward. Now, "love" translates to "liebe." Wait, but in German, the verb conjugation depends on the subject. Since it\'s "I," the verb would be "liebe" because "lieben" is the infinitive, and for first person singular, it\'s "liebe." \n\nNext, "programming" is a gerund in English, which is the -ing form. In German, the equivalent would be the present participle, which is "programmierend." But wait, sometimes in German, they use the noun form instead of the gerund. So maybe it\'s better to say "Ich liebe das Programmieren." Because "Programmieren" is the infinitive noun form, and it\'s commonly used in such contexts. \n\nLet me think again. "I love programming" could be directly translated as "Ich liebe Programmieren," but I\'ve heard both "Programmieren" and "programmierend" used. However, "Ich liebe das Programmieren" sounds more natural because it uses the definite article "das" before the infinitive noun. \n\nAlternatively, if I use "programmieren" without the article, it\'s still correct but maybe a bit less common. So, to make it sound more natural and fluent, including the article "das" would be better. \n\nTherefore, the correct translation should be "Ich liebe das Programmieren." That makes sense because it\'s similar to saying "I love (the act of) programming." \n\nI think that\'s the most accurate and natural way to express it in German. Let me double-check some examples. If someone says "I love reading," in German it\'s "Ich liebe das Lesen." So yes, using "das" before the infinitive noun is the correct structure. \n\nSo, putting it all together, "I love programming" becomes "Ich liebe das Programmieren." That should be the right translation.\n'}, response_metadata={'token_usage': {'completion_tokens': 569, 'prompt_tokens': 18, 'total_tokens': 587, 'completion_time': 2.511255685, 'prompt_time': 0.001466702, 'queue_time': 0.009628211, 'total_time': 2.512722387}, 'model_name': 'deepseek-r1-distill-llama-70b', 'system_fingerprint': 'fp_87eae35036', 'finish_reason': 'stop', 'logprobs': None}, id='run--4d5ee86d-5eec-495c-9c4e-261526cf6e3d-0', usage_metadata={'input_tokens': 18, 'output_tokens': 569, 'total_tokens': 587})

API reference​

For detailed documentation of all ChatGroq features and configurations head to the API reference.