Generative AI continues to push the boundaries of what’s potential. One space garnering vital consideration is the usage of generative AI to research audio and video transcripts, rising our capacity to extract invaluable insights from content material saved in audio or video information. Speech knowledge is exclusive and complicated, which makes it tough to research and extract insights. Manually transcribing and analyzing it may be time-consuming and resource-intensive.

Current strategies for extracting insights from speech knowledge usually require tedious human transcription and evaluation. You need to use computerized voice recognition instruments to transform your audio and video knowledge to textual content. Nonetheless, you continue to should depend on guide processes for extracting particular insights and knowledge factors, or get summaries of the content material. This method is time-consuming and as organizations amass huge quantities of this content material, the necessity for a extra environment friendly and insightful resolution turns into more and more urgent. There’s a vital alternative so as to add enterprise worth given the quantity of information organizations retailer in these codecs and the precious insights which may in any other case go undiscovered. The next are a number of the new insights and capabilities that may be obtained by the usage of massive language fashions (LLM) with audio transcripts:

• LLMs can analyze and perceive the context of a dialog, not simply the phrases spoken, but in addition the implied that means, intent, and feelings. Beforehand, this may have required intensive human interpretation.
• LLMs can carry out superior sentiment evaluation. Beforehand, sentiment evaluation could possibly be captured, however LLMs can seize extra feelings, reminiscent of sarcasm, ambivalence, or blended emotions by understanding the context of the dialog.
• LLMs can generate concise summarizations not simply by extracting content material, however by understanding the context of the dialog.
• Customers can now ask advanced, pure language questions and obtain insightful solutions.
• LLMs can infer personas or roles in a dialog, enabling focused insights and actions.
• LLMs can assist the creation of latest content material based mostly on audio property or conversations following predetermined templates or flows.

On this publish, we look at the right way to create enterprise worth by speech analytics with some examples targeted on the next:

• Routinely summarizing, categorizing, and analyzing advertising content material reminiscent of podcasts, recorded interviews, or movies, and creating new advertising supplies based mostly on these property
• Routinely extracting key factors, summaries, and sentiment from a recorded assembly (reminiscent of an earnings name)
• Transcribing and analyzing contact heart calls to enhance buyer expertise.

Step one in getting these audio knowledge insights includes transcribing the audio file utilizing Amazon Transcribe. Amazon Transcribe is a machine studying (ML) based mostly managed service that robotically converts speech to textual content, enabling builders to seamlessly combine speech-to-text capabilities into their purposes. It additionally acknowledges a number of audio system, robotically redacts personally identifiable info (PII), and permits you to improve the accuracy of a transcription by offering customized vocabularies particular to your industries or use case, or through the use of customized language fashions.

The second step includes utilizing basis fashions (FMs) with Amazon Bedrock to summarize the content material, determine matters, and acknowledge conclusions, extracting invaluable insights that may information strategic choices and improvements. Automated era of latest content material additionally provides worth, rising creativity and productiveness.

Generative AI is reshaping the best way we analyze audio transcripts, enabling you to unlock insights reminiscent of buyer sentiment, ache factors, widespread themes, avenues for threat mitigation, and extra, that had been beforehand obfuscated.

Use case overview

On this publish, we focus on three instance use circumstances intimately. The code artifacts are in Python. We used a Jupyter pocket book to run the code snippets. You’ll be able to comply with alongside by creating and working a pocket book in Amazon SageMaker Studio.

Audio summarization and insights, and automatic era of latest content material utilizing Amazon Transcribe and Amazon Bedrock

Via this use case, we exhibit the right way to take an current advertising asset (a video) and create a brand new weblog publish to announce the launch of the video, create an summary, and extract the primary matters and the SEO (web optimization) key phrases current within the publish for documenting and categorizing the asset.

Transcribe audio with Amazon Transcribe

On this case, we use an AWS re:Invent 2023 technical discuss as a pattern. For the aim of this pocket book, we downloaded the MP4 file for the recording and saved it in an Amazon Easy Storage Service (Amazon S3) bucket.

Step one is to transcribe the audio file utilizing Amazon Transcribe:

# Create a Amazon Transcribe transcirption job by specifying the audio/video file's S3 location
import boto3
import time
import random
transcribe = boto3.consumer('transcribe')
response = transcribe.start_transcription_job(
    TranscriptionJobName=f"podcast-transcription-{int(time.time())}_{random.randint(1000, 9999)}",
    LanguageCode="en-US",
    MediaFormat="mp3",
    Media={
        'MediaFileUri': ''
    },
    OutputBucketName="",
    OutputKey='transcribe_bedrock_blog/output-files/',
    Settings={
        'ShowSpeakerLabels': True,
        'MaxSpeakerLabels': 3
    }
)
max_tries = 60
whereas max_tries > 0:
    max_tries -= 1
    job = transcribe.get_transcription_job(TranscriptionJobName=response['TranscriptionJob']['TranscriptionJobName'])
    job_status = job["TranscriptionJob"]["TranscriptionJobStatus"]
    if job_status in ["COMPLETED", "FAILED"]:
        if job_status == "COMPLETED":
            print(
                f"Obtain the transcript fromn"
                f"t{job['TranscriptionJob']['Transcript']['TranscriptFileUri']}."
            )
        break
    else:
        print(f"Ready for {response['TranscriptionJob']['TranscriptionJobName']}. Present standing is {job_status}.")
    time.sleep(10)

The transcription job will take a couple of minutes to finish.

When the job is full, you may examine the transcription output and examine the plain textual content transcript that was generated (the next has been trimmed for brevity):

# Get the Transcribe Output JSON file
s3 = boto3.consumer('s3')
output_bucket = job['TranscriptionJob']['Transcript']['TranscriptFileUri'].cut up('https://')[1].cut up('/',2)[1]
output_file_key = job['TranscriptionJob']['Transcript']['TranscriptFileUri'].cut up('https://')[1].cut up('/',2)[2]
s3_response_object = s3.get_object(Bucket=output_bucket, Key=output_file_key)
object_content = s3_response_object['Body'].learn()

transcription_output = json.masses(object_content)

# Let's examine what we've contained in the job output JSON
print(transcription_output['results']['transcripts'][0]['transcript'])

……….As soon as the alert comes, how do you form of correlate these alerts, not by simply textual content alerts and textual content passing, however understanding the topology, the infrastructure topology that’s supporting that utility or that enterprise service. It’s the topology that finally provides the supply of fact when an alert comes, proper. That’s what we imply by a correlation that’s assisted with topology in in within the factor that finally leads to discovering a possible root trigger. And as soon as ……….

After you could have validated the existence of the textual content, you need to use Amazon Bedrock to research the output:

# Now let's use this transcript to extract insights with a assist of a Giant Language Mannequin on Amazon Bedrock
# First let's initialize the bedrock runtime consumer to invoke the mannequin. 
bedrock_runtime = boto3.consumer('bedrock-runtime')
# Deciding on Claude 3 Sonnet
model_id = 'anthropic.claude-3-sonnet-20240229-v1:0'

Utilizing the transcription from the technical discuss, we use Amazon Bedrock to name an FM (we use Anthropic’s Claude 3 Sonnet on Amazon Bedrock on this case). You’ll be able to select from the language fashions out there on Amazon Bedrock from AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon.

Now you can carry out extra duties.

Extract the primary matters with Amazon Bedrock

The next immediate gives directions to ask the LLM for the primary matters within the technical discuss:

# Extracting the primary matters with Amazon Bedrock
main_topics_prompt = """Based mostly on the contents of , what are the primary matters being mentioned? Show the matters as an inventory.


{transcript}

"""

user_message = {"position": "person", "content material": main_topics_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
{
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 1000,
"messages": messages
}
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
main_topics = response_body['content'][0]['text']
print(main_topics)

We have now created a immediate that makes use of prompting greatest practices for Anthropic’s Claude. On this case, we go the transcript inside the XML tags and ask for most important matters mentioned. Based mostly on that, we get the next output:

Based mostly on the contents of the transcript, the primary matters being mentioned are:

1. AI and Machine Studying in IT Operations
2. Challenges confronted by IT Operations groups
3. Observable knowledge explosion from cloud, on-premise, and edge infrastructure
4. The convergence of IT, SRE, and DevOps roles
5. Discovery to Decision pipeline for IT Operations
6. Alert correlation and clever alerting utilizing AI/ML
7. Possible root trigger evaluation with AI/ML
8. Automated remediation and runbook automation
9. Predictive analytics and preventive upkeep utilizing AI/ML
10. Advantages of AI/ML for IT Operations (proactive operations, decreased human effort, improved service well being, value optimization)

Generate a weblog publish to announce the video with Amazon Bedrock

The next immediate gives directions to ask the LLM for the era of a brand new weblog publish saying the launch of the previous recording. Identical to the previous immediate, we use prompting greatest practices for Anthropic’s Claude:

# Producing a weblog publish to announce the Podcast with Amazon Bedrock
generate_blog_prompt = """The contents of  are a transcription of a technical discuss video recording. Based mostly on this info write a blogpost that summarizes the technical discuss and proclaims that the recording of this discuss is now out there for public consumption. Restrict the blogpost to 1500 phrases. Don't embody any info that's not talked about in . 


{transcript}

"""

user_message =  {"position": "person", "content material": generate_blog_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
generate_blog = response_body['content'][0]['text']
print(generate_blog)

Based mostly on that, we get the next output:

Here’s a draft 1462 phrase blogpost summarizing the technical discuss and saying the recording is now out there:

Title: Revolutionizing IT Operations with AI/ML: A Future-Prepared Strategy

The world of IT operations is present process a seismic shift, pushed by the explosion of observable knowledge, the convergence of IT roles, and the appearance of hybrid multi-cloud environments. Conventional reactive approaches are not ample to maintain tempo with the complexity and scale of contemporary IT programs. That is the place AI and machine studying (ML) come into play, providing a future-ready method to revolutionize IT operations.

In a current technical discuss, Verma from HPE OpsRamp outlined the drivers behind this AI/ML crucial and demonstrated how this cutting-edge expertise is basically remodeling IT operations. The recording of this insightful session is now out there for public consumption.

The Drivers Behind the AI/ML Crucial

The primary driving power behind the adoption of AI/ML in IT operations is the sheer quantity of observable knowledge that………………..

Create an summary with Amazon Bedrock

The next immediate gives directions to create an summary summarizing the content material of the technical discuss:

# Creating an Summary with Amazon Bedrock
abstract_prompt = """The contents of  are a transcription of a technical discuss video recording. Based mostly on this info write an summary for this technical discuss in not more than 100 phrases. Don't embody any info that's not talked about in .


{transcript}

"""

user_message = {"position": "person", "content material": abstract_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
{
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 1000,
"messages": messages
}
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
summary = response_body['content'][0]['text']
print(summary)

Based mostly on this, we get the next output:

Here’s a 96 phrase summary for the technical discuss:

This discuss mentioned how AI and machine studying improvements are remodeling IT operations for hybrid multi-cloud workloads. The audio system outlined the challenges of coping with exploding observable knowledge from fashionable infrastructure and the convergence of IT roles. They demonstrated how AI/ML methods like clever alerting, alert correlation, possible root trigger evaluation, and automatic remediation can drive extra proactive, predictive operations. Key advantages showcased included decreased human effort, improved service well being, and value optimization. The discuss featured a demo of the OpsRamp platform leveraging AI/ML fashions to streamline the discovery-to-resolution pipeline for managing edge-to-cloud environments.

Extract web optimization key phrases from the generated weblog publish with Amazon Bedrock

The next immediate gives directions to extract the primary web optimization key phrases from the generated weblog publish. Based mostly on on-line analysis on the right way to extract web optimization key phrases from long-form textual content, we got here up with the next checklist. This demonstrates how one can empower an LLM like Anthropic’s Claude to comply with directions and greatest practices for a specific activity or area. Additionally, the immediate specifies that the output ought to be in JSON. That is useful in use circumstances the place you wish to programmatically get outcomes from an LLM and due to this fact require constant formatting. Based mostly on greatest practices for Anthropic’s Claude, we use the Assistant message within the messages API to pre-fill the mannequin’s response to have additional management on the output format:

# Extracting web optimization key phrases from the generated weblog publish
SEO_keywords_prompt = """Extract essentially the most related key phrases and phrases from the given weblog publish textual content current in  that will be invaluable for web optimization (SEO) based mostly on the directions current in  under. The perfect key phrases ought to seize the primary matters, ideas, entities, and high-value phrases current within the content material. Use JSON format with key "key phrases" and worth as an array of key phrases. Skip the preamble; go straight into the JSON. 


{textblog}



1. Fastidiously learn by the whole weblog publish textual content to know the primary matters, ideas, and concepts coated.
2. Establish essential nouns, noun phrases, multi-word phrases, and related adjective-noun combos that relate to the core subject material of the publish.
3. Search for phrases and phrases that potential searchers would possibly use to seek out content material like this.
4. Prioritize phrases which are extremely particular and related to the weblog subject over generic phrases.
5. Range the key phrase size and embody each head phrases (shorter, extra widespread key phrases) and long-tail phrases (longer, extra particular phrases).
6. Goal to extract round 10-20 of essentially the most invaluable, high-impact key phrases and phrases for web optimization.

"""

user_message =  {"position": "person", "content material": SEO_keywords_prompt.format(textblog = generate_blog)}
assistant_message = {"position": "assistant", "content material": '{"key phrases": ['}
messages = [user_message, assistant_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
SEO_keywords = response_body['content'][0]['text']
SEO_keywords="{"key phrases": [" + SEO_keywords
SEO_keywords_json = json.loads(SEO_keywords)
print(SEO_keywords_json)

Based on this, we get the following output:

{‘keywords’: [‘AI-driven IT operations’, ‘machine learning IT operations’, ‘proactive IT operations’, ‘predictive IT operations’, ‘AI for hybrid cloud’, ‘AI for multi-cloud’, ‘FutureOps’, ‘AI-assisted IT operations’, ‘AI-powered event correlation’, ‘intelligent alerting’, ‘automated remediation workflows’, ‘predictive analytics for IT’, ‘AI anomaly detection’, ‘AI root cause analysis’, ‘AI-driven observability’, ‘AI for DevOps’, ‘AI for SRE’, ‘AI IT operations management’]}

For constant formatting and structured output, you can too use the Converse and ConverseStream APIs in Amazon Bedrock and use the device calling capabilities of the LLMs that provide it.

Generate a brand new weblog publish model emphasizing particular web optimization key phrases with Amazon Bedrock

The next immediate gives directions to generate a brand new weblog publish emphasizing three key phrases or phrases from those extracted within the earlier step. On this case, we use the primary three key phrases within the returned key phrases checklist. The directions specified within the immediate are derived from web optimization content material greatest practices:

# Extracting web optimization key phrases from the generated weblog publish
rewrite_SEO_keywords_prompt = """Rewrite the given weblog publish textual content current in  to enhance its rating for 3 goal key phrases listed in . Rewrite the weblog publish, naturally incorporating the goal key phrases a number of instances all through the textual content. Hold the rewritten content material high-quality, readable, and related, whereas strategically emphasizing the key phrases to enhance web optimization.


{textblog}



{kw1},
{kw2},
{kw3}

"""

user_message = {"position": "person", "content material": rewrite_SEO_keywords_prompt.format(textblog = generate_blog, kw1 = SEO_keywords_json['keywords'][0], kw2 = SEO_keywords_json['keywords'][1], kw3 = SEO_keywords_json['keywords'][2])}
messages = [user_message]

physique=json.dumps(
{
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 1000,
"messages": messages
}
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
rewrite_SEO_blog = response_body['content'][0]['text']
print(rewrite_SEO_blog)

Based mostly on this, we get the next output:

Title: Unleash the Energy of AI-Pushed and Machine Studying IT Operations for Proactive IT Operations

In right now’s quickly evolving digital panorama, conventional IT operations approaches are struggling to maintain tempo. The explosion of information from cloud workloads, Kubernetes clusters, and distributed edge areas has overwhelmed standard monitoring instruments, resulting in fragmented views and reactive operations that reply to points after they happen. To remain forward of the curve, organizations should embrace AI-driven and machine studying IT operations, paving the best way for proactive IT operations that improve effectivity and reduce downtime.

At a current technical discuss, Verma and Radha Krishna Kunni from OpsRamp (not too long ago acquired by HPE) delved into the transformative influence of AI and machine studying on IT operations, DevOps, and SRE for hybrid multi-cloud environments. They highlighted the important thing challenges ops groups face right now and launched the revolutionary “FutureOps” method, which leverages AI and machine studying to revolutionize IT operations.

The total video recording of this insightful technical discuss is now out there https://aws.amazon.com/blogs/machine-learning/unearth-insights-from-audio-transcripts-generated-by-amazon-transcribe-using-amazon-bedrock/, offering a complete understanding of…………

Summarize content material mentioned in a recorded assembly utilizing Amazon Transcribe and Amazon Bedrock

Via this use case, we exhibit the right way to take an current recording from a gathering (we use a recording from an AWS earnings name) to summarize the content material mentioned, extract the important thing factors, and supply particulars on the sentiment of the assembly. For added info on this use case, see Stay Assembly Assistant with Amazon Transcribe, Amazon Bedrock, and Amazon Bedrock Information Bases or Amazon Q Enterprise.

Transcribe audio with Amazon Transcribe

On this use case, we use an Amazon 2024 Q1 earnings name as a pattern. For the aim of this pocket book, we downloaded the WAV file for the recording and saved in an S3 bucket.

Step one is to transcribe the audio file utilizing Amazon Transcribe:

# Create a Amazon Transcribe transcription job by specifying the audio/video file's S3 location
import boto3
import time
import random
transcribe = boto3.consumer('transcribe')
response = transcribe.start_transcription_job(
    TranscriptionJobName=f"meeting-transcription-{int(time.time())}_{random.randint(1000, 9999)}",
    LanguageCode="en-US",
    MediaFormat="mp3",
    Media={
        'MediaFileUri': ''
    },
    OutputBucketName="",
    OutputKey='transcribe_bedrock_blog/output-files/',
    Settings={
        'ShowSpeakerLabels': True,
        'MaxSpeakerLabels': 10
    }
)
# Test whether or not the transcribe job is full

max_tries = 60
whereas max_tries > 0:
    max_tries -= 1
    job = transcribe.get_transcription_job(TranscriptionJobName=response['TranscriptionJob']['TranscriptionJobName'])
    job_status = job["TranscriptionJob"]["TranscriptionJobStatus"]
    if job_status in ["COMPLETED", "FAILED"]:
        if job_status == "COMPLETED":
            print(
                f"Obtain the transcript fromn"
                f"t{job['TranscriptionJob']['Transcript']['TranscriptFileUri']}."
            )
        break
    else:
        print(f"Ready for {response['TranscriptionJob']['TranscriptionJobName']}. Present standing is {job_status}.")
    time.sleep(10)

The transcription job will take a couple of minutes to finish.

When the job is full, you may examine the transcription output and examine for the plain textual content transcript that was generated:

import json
# Get the Transcribe Output JSON file
s3 = boto3.consumer('s3')
output_bucket = job['TranscriptionJob']['Transcript']['TranscriptFileUri'].cut up('https://')[1].cut up('/',2)[1]
output_file_key = job['TranscriptionJob']['Transcript']['TranscriptFileUri'].cut up('https://')[1].cut up('/',2)[2]
s3_response_object = s3.get_object(Bucket=output_bucket, Key=output_file_key)
object_content = s3_response_object['Body'].learn()

transcription_output = json.masses(object_content)

# Let's examine what we've contained in the job output JSON
print(transcription_output['results']['transcripts'][0]['transcript'])

Thanks for standing by. Good day, everybody and welcome to the amazon.com first quarter, 2024 monetary outcomes teleconference. At the moment, all contributors are in a pay attention solely mode. After the presentation, we are going to conduct a query and reply session. Right now’s name is being recorded and for opening remarks, I’ll be turning the decision over to the Vice President of Investor………

After you could have validated the existence of the textual content, you need to use Amazon Bedrock to research the output:

# Now let's use this transcript to extract insights with a assist of a Giant Language Mannequin on Amazon Bedrock
# First let's initialize the bedrock runtime consumer to invoke the mannequin. 
bedrock_runtime = boto3.consumer('bedrock-runtime')
# Deciding on Claude 3 Sonnet
model_id = 'anthropic.claude-3-sonnet-20240229-v1:0'

Utilizing the transcription from the earnings name recording, we use Amazon Bedrock to name an FM (we use Anthropic’s Claude 3 Sonnet on this case). You’ll be able to select from different FMs out there on Amazon Bedrock.

Now you can carry out extra duties.

Establish the monetary ratios highlighted throughout this earnings name

The next immediate gives directions to determine monetary ratios highlighted throughout the earnings name and their implications:

# Establish the monetary ratios highlighted throughout this earnings name 
financial_ratios_prompt = """Based mostly on the contents of ,determine the monetary ratios highlighted throughout this earnings name and their implications  . 


{transcript}

"""

user_message =  {"position": "person", "content material": financial_ratios_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
financial_ratios = response_body['content'][0]['text']
print(financial_ratios)

Based mostly on this, we get the next output:

Based mostly on the earnings name transcript, the next monetary ratios and their implications had been highlighted:

1. Working Revenue Margin:
   • Amazon reported its highest ever quarterly working revenue of $15.3 billion, which was $3.3 billion above the excessive finish of their steerage vary. This was pushed by robust operational efficiency throughout all three reportable segments (North America, Worldwide, and AWS) and better-than-expected working leverage, together with decrease value to serve.
   • North America phase working revenue was $5 billion with an working margin of 5.8%, up 460 foundation factors year-over-year, pushed by enhancements in value to serve, together with advantages from regionalization efforts, extra consolidated buyer shipments, and improved leverage.
   • Worldwide phase working revenue was $903 million with an working margin of two.8%, up 710 foundation factors year-over-year, primarily pushed by value efficiencies by community design enhancements and improved quantity leverage in established international locations, in addition to progress in rising international locations.
   • AWS working revenue was $9.4 billion, a rise of $4.3 billion year-over-year, with improved leverage from managing infrastructure and glued prices whereas rising at a wholesome charge.

Implication: The upper working revenue margins throughout all segments point out Amazon’s give attention to driving efficiencies and bettering profitability whereas persevering with to spend money on development alternatives.

2. Income Development:
   • Worldwide income was $143.3 billion, up 13% year-over-year (excluding the influence of international alternate).
   • AWS income grew 17.2% year-over-year, accelerating from 13.2% in This autumn 2023, pushed by robust demand for each generative AI and non-generative AI workloads.
   • Promoting income grew 24% year-over-year (excluding the influence of international alternate), primarily pushed by sponsored merchandise and enhancements in relevancy and measurement capabilities.

Implication: The robust income development, significantly in AWS and promoting, highlights Amazon’s diversified income streams and the expansion alternatives in cloud computing and digital promoting.

3. Capital Expenditures (Capex):
   • Amazon anticipates a significant enhance in general capital expenditures in 2024, primarily pushed by increased infrastructure Capex for development in AWS, together with generative AI investments.
   • In Q1 2024, Capex was $14 billion, anticipated to be the bottom quarter of the yr.

Implication: The rise in Capex alerts Amazon’s confidence within the robust demand for AWS and their dedication to investing in rising applied sciences like generative AI to drive future development.

General, the monetary ratios and commentary point out Amazon’s give attention to bettering profitability, driving operational efficiencies, and investing in development alternatives, significantly in AWS and generative AI, whereas sustaining a diversified income stream and managing prices successfully.

Establish the audio system from the earnings name with Amazon Bedrock

The next immediate gives directions to determine the audio system within the assembly from the transcription:

# Establish the audio system from the earnings name 
speakers_prompt = """Based mostly on the contents of ,determine the audio system on this earnings name  . 


{transcript}

"""

user_message =  {"position": "person", "content material": speakers_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
audio system = response_body['content'][0]['text']
print(audio system)

Based mostly on this, we get the next output:

Based mostly on the transcript, the important thing audio system on this Amazon earnings name seem like:

1. Andy Jassy – CEO of Amazon
2. Brian Olsavsky – CFO of Amazon
3. Dave Fildes – Vice President of Investor Relations at Amazon

The decision begins with opening remarks from Dave Fildes, adopted by ready statements from Andy Jassy and Brian Olsavsky. They then take questions from analysts, with Andy and Brian offering the responses.

Receive the challenges or unfavourable areas mentioned on the earnings name with Amazon Bedrock

The next immediate gives directions to acquire the challenges or unfavourable areas mentioned from the transcription:

# Receive the challenges or unfavourable areas mentioned on earnings name

challenges_prompt = """Based mostly on the contents of ,Acquiring the challenges or unfavourable areas mentioned on earnings  . 


{transcript}

"""

user_message =  {"position": "person", "content material": challenges_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
challenges = response_body['content'][0]['text']
print(challenges)

Based mostly on this, we get the next output:

Based mostly on the transcript, a number of the key challenges or unfavourable areas mentioned embody:

1. International alternate headwinds: Amazon confronted an unfavorable influence from world currencies weakening towards the U.S. greenback in Q1, resulting in a $700 million or 50 foundation level headwind to income in comparison with steerage.
2. Growing capital expenditures: Amazon expects to meaningfully enhance its capital expenditures year-over-year in 2024, primarily pushed by increased infrastructure spending for AWS development, together with investments in generative AI capabilities.
3. Client spending issues: Amazon talked about maintaining a tally of shopper spending tendencies, particularly in Europe, the place it seems weaker relative to the U.S.
4. Worldwide phase profitability: Whereas the worldwide phase’s profitability improved, with an working margin of two.8%, Amazon acknowledged the necessity to proceed engaged on value efficiencies and profitability, significantly in rising international locations.
5. Value optimization challenges: Though Amazon believes nearly all of value optimization efforts are behind them, there may be nonetheless a necessity to repeatedly streamline processes, optimize stock placement, and spend money on automation to additional cut back the fee to serve.

General, the challenges centered round international alternate impacts, rising capital depth for AWS and generative AI investments, shopper demand uncertainties, and ongoing efforts to enhance operational efficiencies and worldwide profitability.

Get insights from a name heart name between an agent and a buyer utilizing Amazon Transcribe and Amazon Bedrock

Via this use case, we exhibit the right way to take an current name recording from a contact heart and summarize the content material mentioned, extract the primary subject, key phrases, name cause, buyer satisfaction, general name sentiment, and sentiment concerning the services and products mentioned. For added particulars about this use case, see Stay name analytics and agent help on your contact heart with Amazon language AI providers and Put up name analytics on your contact heart with Amazon language AI providers.

Transcribe audio with Amazon Transcribe

Step one is to transcribe the audio file utilizing Amazon Transcribe. On this case, we use a pattern from the Amazon Transcribe Put up Name Analytics Answer GitHub repository. For the aim of this pocket book, we downloaded the WAV file and saved it in an S3 bucket.

# Create a Amazon Transcribe transcirption job by specifying the audio/video file's S3 location
import boto3
import time
import random
transcribe = boto3.consumer('transcribe')
response = transcribe.start_transcription_job(
    TranscriptionJobName=f"call_center-transcription-{int(time.time())}_{random.randint(1000, 9999)}",
    LanguageCode="en-US",
    MediaFormat="mp3",
    Media={
        'MediaFileUri': ''
    },
    OutputBucketName="",
    OutputKey='transcribe_bedrock_blog/output-files/',
    Settings={
        'ShowSpeakerLabels': True,
        'MaxSpeakerLabels': 3
    }
)
max_tries = 60
whereas max_tries > 0:
    max_tries -= 1
    job = transcribe.get_transcription_job(TranscriptionJobName=response['TranscriptionJob']['TranscriptionJobName'])
    job_status = job["TranscriptionJob"]["TranscriptionJobStatus"]
    if job_status in ["COMPLETED", "FAILED"]:
        if job_status == "COMPLETED":
            print(
                f"Obtain the transcript fromn"
                f"t{job['TranscriptionJob']['Transcript']['TranscriptFileUri']}."
            )
        break
    else:
        print(f"Ready for {response['TranscriptionJob']['TranscriptionJobName']}. Present standing is {job_status}.")
    time.sleep(10)

The transcription job will take a couple of minutes to finish.

When the job’s full, you may examine the transcription output and examine for the plain textual content transcript that was generated:

import json
# Get the Transcribe Output JSON file
s3 = boto3.consumer('s3')
output_bucket = job['TranscriptionJob']['Transcript']['TranscriptFileUri'].cut up('https://')[1].cut up('/',2)[1]
output_file_key = job['TranscriptionJob']['Transcript']['TranscriptFileUri'].cut up('https://')[1].cut up('/',2)[2]
s3_response_object = s3.get_object(Bucket=output_bucket, Key=output_file_key)
object_content = s3_response_object['Body'].learn()

transcription_output = json.masses(object_content)

# Let's examine what we've contained in the job output JSON
print(transcription_output['results']['transcripts'][0]['transcript'])

Thanks for calling Large Jim’s Auto. That is Travis. How can I allow you to right now? Hey, my title is Violet King and I purchased a automobile not too way back and a lightweight is approaching um a lightweight on the dashboard. And so I used to be questioning what I ought to do about that. Okay. It might rely on what sort of mild we’re right here right now, ma’am. Uh Might I get your first and final title spelled out for me so I can simply get some info pulled up? Sure. My title is Violet Vviolet. My final title is King King. Okay, I received the decision final week. Okay. Uh And what sort of automobile are we analyzing right now? It’s, it’s a Ford Fusion. It’s 2017, 2017 Ford Fusion. OK. And for verification, ma’am. Do you occur to know the acquisition date of the automobile? Sure, it, it, it was final Tuesday, August tenth. You say the tenth? Okay. And might you describe to me what sort of mild we’re ? Y sure, it’s uh I, I believe it’s a, an oil, an oil mild, an oil mild? Okay. Okay. And uh only for readability on my finish, ma’am. Um, uh, is that this the primary name you’ve made concerning this? Sure. Okay. And, uh, this is likely to be form of a foolish query as a result of I do know you simply received the automobile. However typically they make me ask a foolish query about what number of miles has the automobile been pushed since to procure it? Oh. I’m undecided. Ought to I examine? Uh, no, that’s okay. I’ll simply, I’ll simply put in that. We don’t know at the moment. It’s okay. Um, so, um, uh, beneath the guarantee we provide, um, we, the, we don’t deal with in home oil adjustments. Um, we principally, when, when somebody buys a automobile from us, the guarantee, we’ve, it, it covers some stuff like, um, climate harm. Um, and, uh, if the engine mild comes on, we check out that, however the oil change is one thing that we simply don’t have, uh, right here on the dealership that’s just a little bit outsourced out they usually’re fairly backed up proper now as a result of lots of people have been, uh, staying in because of the current pandemic and now everybody’s simply beginning to get out and a complete bunch of locations are simply fully slowed down. So we’ve a spot that we usually outsource to, um, they usually’re, they’re fairly cheap. They’re about, I wanna say someplace between 25 and $35 to do an oil change. So it’s actually not that dangerous, however they’re just a little bit backed up proper now from what I’ve heard, I might suggest giving them a name as quickly as you may earlier than they shut. Okay. What’s their quantity? Uh, give me a second. Let me simply rustle by the desk right here. See if I can discover their info. Uh, okay. Okay. Sure, I’m all proper with them one second, please. Certain. Okay. All their quantity is 888 333 2222. Okay, they usually can repair my automobile. Yeah, they need to be capable to deal with the oil change. I’m sorry, that’s not one thing that we cowl beneath the guarantee that uh, we’ve, um, however they need to be capable to get you settled and, uh, sorted. Okay. Okay. Thanks. Cool. No drawback. Have a very good one. Thanks. Yup. Bye.

After you could have validated the existence of the textual content, you need to use Amazon Bedrock to research the output:

# Now let's use this transcript to extract insights with a assist of a Giant Language Mannequin on Amazon Bedrock
# First let's initialize the bedrock runtime consumer to invoke the mannequin. 
bedrock_runtime = boto3.consumer('bedrock-runtime')
# Deciding on Claude 3 Sonnet
model_id = 'anthropic.claude-3-sonnet-20240229-v1:0'

Utilizing the transcription from the recorded name heart name, we use Amazon Bedrock to name an FM (we use Anthropic’s Claude 3 Sonnet on this case), however you may select from the opposite language fashions out there on Amazon Bedrock.

Now you can carry out extra duties.

Summarize the decision between agent and consumer with Amazon Bedrock

The next immediate gives directions to summarize the decision dialogue from the transcription:

#Summarize the decision between agent and buyer
summarize_prompt = """Based mostly on the contents of ,summarize the decision between agent and buyer with give attention to decision  . 


{transcript}

"""

user_message =  {"position": "person", "content material": summarize_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
summarization = response_body['content'][0]['text']
print(summarization)

We get the next output:

Based mostly on the contents of the transcript, here’s a abstract of the decision between the agent (Travis) and the client (Violet King) with a give attention to decision:

Violet King referred to as a couple of mild on the dashboard of her not too long ago bought 2017 Ford Fusion from Large Jim’s Auto. The sunshine seemed to be an oil mild. Travis defined that whereas their guarantee covers sure points like climate harm and engine lights, it doesn’t cowl oil adjustments. He beneficial calling an outsourced oil change service that Large Jim’s Auto usually makes use of, which prices between $25-35 for an oil change.

Travis offered the telephone quantity for the oil change service (888-333-2222) and talked about that they’re presently backed up because of the current pandemic. He suggested Violet to name them as quickly as potential earlier than they near get her oil modified and resolve the problem with the oil mild on her dashboard.

The decision was for Violet to contact the beneficial third-party oil change service to have her automobile’s oil modified, which ought to deal with the oil mild challenge she was experiencing along with her newly bought automobile.

Extract the primary matters with Amazon Bedrock

The next immediate gives directions to extract the primary matters mentioned within the dialog from the transcription:

# Extracting the primary matters from the dialog
maintopic_prompt = """The contents of  are a transcription of a dialog between agent and consumer. Based mostly on the data, extract the primary matters from the dialog  . 


{transcript}

"""

user_message =  {"position": "person", "content material": maintopic_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
main_topic = response_body['content'][0]['text']
print(main_topic)

We get the next output:

Based mostly on the dialog transcript, the primary matters seem like:

1. Dashboard warning mild (particularly an oil mild) on a not too long ago bought 2017 Ford Fusion.
2. Figuring out if the problem is roofed beneath the guarantee offered by the dealership (Large Jim’s Auto).
3. Suggestion to contact an exterior auto service supplier (telephone quantity offered) for an oil change service, because the dealership doesn’t deal with oil adjustments in-house.
4. Confirming that the exterior auto service supplier can probably resolve the oil mild challenge by performing an oil change.

Extract the important thing phrases with Amazon Bedrock

The next immediate gives directions to extract the important thing phrases mentioned within the dialog from the transcription:

# Extracting the important thing phrases with Amazon Bedrock
keyphrase_prompt = """The contents of  are a transcription of a dialog between agent and consumer. Based mostly on the data, extract the important thing phrases mentioned within the dialog . 


{transcript}

"""

user_message =  {"position": "person", "content material": keyphrase_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
keyphrases = response_body['content'][0]['text']
print(keyphrases)

We get the next output:

Based mostly on the dialog transcript, listed here are the important thing phrases mentioned:

• Oil mild
• 2017 Ford Fusion
• Buy date: August tenth
• First name concerning the problem
• Automobile mileage unknown
• Guarantee doesn’t cowl oil adjustments
• Outsourced oil change service beneficial
• Oil change service contact quantity: 888-333-2222
• Oil change service value: $25 – $35
• Service is backed up because of the pandemic

Extract the explanation why the consumer referred to as the decision heart with Amazon Bedrock

The next immediate gives directions to extract the explanation for this consumer name to the decision heart from the transcription:

# Extracting the explanation why consumer referred to as the decision heart
reason_prompt = """The content material of  is  transcription of a dialog between agent and consumer. Based mostly on the data, extract the explanation why consumer referred to as the decision heart . 


{transcript}

"""

user_message =  {"position": "person", "content material": reason_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
cause = response_body['content'][0]['text']
print(cause)

We get the next output:

Based mostly on the transcription, the consumer referred to as the decision heart as a result of a lightweight (particularly an oil mild) was approaching the dashboard of their not too long ago bought 2017 Ford Fusion automobile. The consumer was looking for steerage on what to do concerning the oil mild being on.

Extract the extent of buyer satisfaction with Amazon Bedrock

The next immediate gives directions to extract the extent of buyer satisfaction skilled by the consumer from the transcription:

# Extracting the extent of Buyer Satisfaction 
satisfaction_prompt = """The content material of  is  transcription of a dialog between agent and consumer. Based mostly on the data, extract the extent of buyer satisfaction . 


{transcript}

"""

user_message =  {"position": "person", "content material": satisfaction_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
csat= response_body['content'][0]['text']
print(csat)

We get the next output:

Based mostly on the transcript, the extent of buyer satisfaction appears to be average.

Proof:

1. The agent offered clear explanations concerning the problem with the oil mild and why oil adjustments aren’t coated beneath their guarantee.
2. The agent provided a suggestion for an exterior service supplier that might carry out the oil change, together with their contact info.
3. The shopper acknowledged the data offered by the agent, indicating some stage of satisfaction with the response.

Nonetheless, there aren’t any express statements from the client expressing excessive satisfaction or dissatisfaction. The interplay stays well mannered and resolves the client’s preliminary question, however there isn’t any robust indication of remarkable satisfaction or disappointment.

Receive the general buyer sentiment with Amazon Bedrock

The next immediate gives directions to acquire the general buyer sentiment from the transcription:

# Extracting the general buyer sentiment.
sentiment_prompt = """The content material of  is transcription of a dialog between agent and consumer. Based mostly on the data, what's the general buyer sentiment of the dialog . 


{transcript}

"""

user_message =  {"position": "person", "content material": sentiment_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
sentiment = response_body['content'][0]['text']
print(sentiment)

We get the next output:

Based mostly on the dialog transcript, the general buyer sentiment appears impartial to barely optimistic. Though the client, Violet King, was initially involved a couple of warning mild on her not too long ago bought automobile’s dashboard, the agent (Travis) defined the scenario clearly and offered a suggestion for getting an oil change from a third-party service supplier. The shopper acknowledged and accepted the suggestion with out expressing vital frustration or dissatisfaction. The dialog ended on a well mannered notice with the client thanking the agent.

Receive sentiment concerning the services or products mentioned throughout the name with Amazon Bedrock

The next immediate gives directions to acquire sentiment about services and products mentioned throughout the name from the transcription:

## Acquiring sentiment concerning the services or products mentioned.
sentiment_product_prompt = """The content material of  is transcription of a dialog between agent and consumer. Based mostly on the data, what's the general sentiment concerning the merchandise mentioned within the dialog . 


{transcript}

"""

user_message =  {"position": "person", "content material": sentiment_product_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
sentiment_product = response_body['content'][0]['text']
print(sentiment_product)

We get the next output:

Based mostly on the dialog transcript, the general sentiment concerning the merchandise (the 2017 Ford Fusion automobile) mentioned appears impartial to barely optimistic. There aren’t any main complaints or robust unfavourable sentiments expressed concerning the automobile itself. The difficulty raised is concerning an oil mild approaching, which appears to be a routine upkeep matter that must be addressed by getting an oil change. The agent helpfully gives the client with info on the place to get an oil change performed, though mentioning that the guarantee doesn’t cowl routine oil adjustments on the dealership. General, there isn’t any indication of main dissatisfaction with the automobile itself from the dialog.

Create an e mail to ship to the consumer with the decision abstract

The next immediate gives directions to create an e mail to ship to the consumer after the decision is accomplished, summarizing the decision:

## Creating an e mail to ship to the consumer with the decision abstract
email_prompt = """The content material of  is transcription of a dialog between agent and consumer. Based mostly on the data, create an e mail to ship to the consumer after the decision is accomplished summarizing the decision. 
Person formal however pleasant english model. . 


{transcript}

"""

user_message =  {"position": "person", "content material": email_prompt.format(transcript = transcription_output['results']['transcripts'][0]['transcript'])}

messages = [user_message]

physique=json.dumps(
    {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1000,
        "messages": messages
    }
)
bedrock_response = bedrock_runtime.invoke_model(physique=physique, modelId=model_id)
response_body = json.masses(bedrock_response.get('physique').learn())
email_conversation = response_body['content'][0]['text']
print(email_conversation)

We get the next output:

Expensive Ms. Violet King,

I hope this e mail finds you effectively. I’m writing to summarize our dialog concerning the problem together with your 2017 Ford Fusion that you simply bought final Tuesday, August tenth.

Throughout our name, you talked about that an oil mild was illuminated on the dashboard of your automobile. As mentioned, our dealership’s guarantee doesn’t cowl routine upkeep providers like oil adjustments. Nonetheless, we’ve a trusted companion that we usually suggest for such providers.

The beneficial auto service supplier is reachable at 888-333-2222. They’re skilled in dealing with oil adjustments and may be capable to help you together with your automobile’s wants. Please notice that they’ve been experiencing a excessive quantity of requests because of the current pandemic, so it’s advisable to name them as quickly as potential to schedule an appointment.

Conclusion

Utilizing generative AI by Amazon Bedrock to research audio transcripts generated by Amazon Transcribe unlocks invaluable insights that will in any other case stay hidden inside the audio knowledge. By combining the highly effective speech-to-text capabilities of Amazon Transcribe with the pure language understanding and era capabilities of LLMs like these out there by Amazon Bedrock, companies can extra effectively extract key info, generate summaries, determine matters and sentiments, and create new content material from their audio and video property. This method not solely saves time and assets in comparison with guide transcription and evaluation, but in addition opens up new alternatives for utilizing current content material in revolutionary methods.

Whether or not it’s repurposing advertising supplies, shortly capturing key factors from conferences, or bettering buyer expertise by name heart analytics, the mixture of Amazon Transcribe and huge language fashions (LLMs) on Amazon Bedrock gives a strong resolution for unlocking the complete potential of audio knowledge.As these use circumstances have demonstrated, this expertise will be utilized throughout varied domains, from content material creation and web optimization optimization to enterprise intelligence and customer support. By staying on the forefront of those developments, organizations can acquire a aggressive edge by successfully harnessing the wealth of knowledge contained inside their audio and video repositories, driving insights, and making extra knowledgeable choices.

In regards to the Authors

Ana Maria Echeverri is an AI/ML Worldwide Service Specialist at AWS, targeted on driving adoption of generative AI speech analytics use circumstances. She has labored within the knowledge and AI business for over 30 years, with over 10 years targeted on serving to organizations develop their AI maturity and capabilities for profitable execution of AI methods.

Vishesh Jha is a Senior Options Architect at AWS. His space of curiosity lies in generative AI, and he has helped prospects and companions get began with NLP utilizing AWS providers reminiscent of Amazon Bedrock, Amazon Transcribe, and Amazon SageMaker. He’s an avid soccer fan, and in his free time enjoys watching and enjoying the game. He additionally loves cooking, gaming, and touring along with his household.

Bala Krishna Jakka is a Technical Account supervisor at AWS, with a ardour for contact heart and generative AI applied sciences. With intensive experience in serving to organizations use cutting-edge options, he thrives on staying forward of the curve on this quickly evolving subject. When not immersed within the realms of AI and buyer expertise, he finds pleasure within the recreation of cricket, showcasing his expertise on the pitch. A faithful household man, he cherishes the moments spent along with his family members, creating lasting recollections and discovering stability amidst the calls for of his skilled pursuits