SmartThings Safe Premium is powered by Arlo, a third-party partner that operates the professional monitoring center and coordinates emergency dispatch on our behalf.

Feeling safe, whether at home or on the go, shouldn’t take more than a tap. With SmartThings Safe Premium, users can instantly send help requests, from inside the SmartThings app, to Arlo-powered emergency response providers who can send police, fire, or medical help to their exact location.

SmartThings Safe includes a Basic plan at no cost; the Premium plan adds 24/7 professional emergency dispatch through Arlo.

How It Works

When you press the Safe button, a 10-second countdown starts. After 10 seconds, a help request is automatically sent to a professional monitoring center.

• An Arlo monitoring agent will immediately call and text you to confirm what type of help (fire, police, or medical) is needed.
•  If they can’t reach you, they’ll request that local emergency responders (police by default) be dispatched to your current GPS location.
• At the same time, just like the basic plan, your Location Members will receive a push notification with a map of your location and options to call or text you directly.

You can cancel your request at any time during the 10-second countdown. Once the alert is created, you can still cancel the request by responding to the monitoring center’s call or text.

Setting Up SmartThings Safe Premium 

Getting started takes just a few steps:

• Open the SmartThings app on your mobile device.
• Tap the Menu icon at the bottom right of the SmartThings app.
• Select Safe and complete setup, including agreeing to the terms agreement and completing identity verification.
• Go to My Plan and select Upgrade to Premium Plan.
• Follow the Samsung Checkout steps to activate your limited-time free access to SmartThings Safe Premium.
• Once activated, you’ll see your Safe button appear in the app, ready whenever you need it. 

Share alerts with the people who matter. Add family or friends to any SmartThings location — home, office, wherever — and they’ll stay in the loop automatically. 

• From the Home tab, tap the three-dot menu in the upper-right corner.
• Select Location Settings > Invite Members.
• Enter the Samsung account email addresses of the people you want to add.

Added members will need the following to receive Safe alerts:

• Have the SmartThings app downloaded.
• Set up their own Samsung account.
• Tap the Menu icon at the bottom right of the SmartThings app.
• Tap the Safe icon and follow the prompts to get started.
• Review and agree to the Terms & Conditions and complete two-factor authentication. 
• Have notifications turned on to receive Safe alerts.

These members will then become emergency contacts for dispatchers and the monitoring center.

For even faster access, you can add the Safe button to your Galaxy home screen or Quick Panel by selecting Settings > Add to Home Screen from the device card.

Make Sure Location Services Are On

To share your real-time GPS location with the monitoring center and your added members using the Safe Premium plan, you’ll need to enable Location Services on your phone.

•  On Galaxy and Android devices: Go to Settings > Location > App permissions > SmartThings > Allow all the time.
• On iOS: Go to Settings > Privacy & Security > Location Services > SmartThings > Always and turn on Precise Location.

Safe Premium is designed to work when your mobile device has a signal. The service is currently available in the U.S. for supported Galaxy, Android, and iOS devices. Both the Basic and Premium plans of SmartThings Safe are designed for manual, user-triggered help requests, making them a great complement to built-in SOS features on smartphones. While Safe Premium does not currently support automations, routines, or smartwatch-triggered requests, users can receive Safe notifications on Galaxy Watches and Apple Watches.

Whether you’re checking in with loved ones or reaching out for help, SmartThings makes it simple to stay safe, connected, and in control.

Activate your *limited-time free access to SmartThings Safe Premium in the SmartThings app today. 

*SmartThings Safe Premium is currently available as a beta service. Free access may be changed or discontinued without notice.

A recognition budget isn’t about finding the “right” number off a benchmark chart. It’s about building a number tied to a program design that’s actually going to drive outcomes—one that doesn’t just acknowledge people, but changes behavior, surfaces performance signals, and makes culture visible every day.

Most organizations invest in recognition and never see that return. The program exists. The intention is real. But recognition stays slow, manager-dependent, or stripped of rewards—and people don’t feel valued. Feeling valued is foundational to thriving teams, but a budget alone won’t get you there. The design has to be right first.

That’s what this blog is about: building the number and the program behind employee recognition—one you can defend to finance or leadership.

What is an employee recognition budget? 

An employee recognition budget is the dedicated annual spend an organization commits to acknowledging and rewarding its employees. It funds everything from peer shoutouts and manager-led awards to formal milestones and redeemable rewards. Without a defined budget, recognition stays reactive—something that happens when someone remembers, rather than something built into how your organization runs. Our research shows programs that pair recognition with rewards consistently outperform those that rely on acknowledgment alone.

How much should you spend per employee? 

Impactful recognition programs can start as low as $5 per employee per month—that’s $60 per person annually, or $30,000 for a 500-person team. Organizations with the strongest recognition cultures typically spend closer to $50–$60 per person per month, layering in peer-to-peer recognition, manager-led awards, formal milestones, and a rewards catalog employees can actually choose from.

The gap between $5 and $60 isn’t about company size. It’s about program depth and employee choice.

What matters more than hitting a specific number is building an employee recognition budget that’s recurring and predictable. Ad hoc recognition produces exactly the kind of inconsistency that kills program credibility. Finance needs a line item. Managers need an allocation. Employees need to know recognition is part of how the organization works—not a surprise when someone remembers.

Why the “1% of payroll” benchmark isn’t enough

You’ve probably heard it: allocate 1% of payroll to recognition. It’s not a bad starting point—but it isn’t yours.

A generic benchmark doesn’t account for your headcount, your culture goals, or the types of recognition you actually want to run. And a number that isn’t specific to your organization won’t land with finance the way a customized budget will.

That’s the real problem most HR leaders face. It’s not a lack of belief in recognition. It’s the absence of a number they can trust.

How to find out your current employee recognition spend 

Before you can build a recognition budget, you need to know what you’re already spending—because most organizations are informally spending more than they think. It’s just scattered across expense reports, procurement orders, and manager credit cards, with no one connecting the dots.

Here’s how to get the real number in four steps.

1. Pull a 12-month expense report look-back. Ask your finance team to filter transactions by Meals & Entertainment, Amazon Business, Gifts, Promotional Items, Office Supplies over $200, and gift card purchases. You’re looking for patterns: the same manager spending $50–$200 a month, recurring Amazon orders, Q4 spikes. Those aren’t anomalies—they’re your informal recognition program.
2. Talk to procurement. Any company buying branded swag or logo merchandise for internal use—onboarding kits, sales kick-offs, service anniversaries—has a vendor relationship. Pull the last 12 months of POs from those suppliers. Most HR leaders are genuinely surprised by what adds up.
3. Check your corporate card merchant codes. MCC codes 5812 (restaurants), 5999 (misc retail), 5045 (gift/novelty), and 7299 (misc services) are where informal recognition spend tends to hide. A one-hour export from your Amex or Visa business card data tells the story faster than any internal survey.
4. Ask your managers directly. A two-question anonymous survey goes a long way: “In the last 90 days, how much did you personally spend—out-of-pocket or on your company card — to recognize your team?” and “What did you buy?” The answers are almost always eye-opening. Managers are already spending. They just aren’t doing it in a way that’s visible, consistent, or connected to culture.

Once you have that number, the conversation with finance changes. You’re not asking for new spend—you’re asking to consolidate what’s already happening into an employee recognition program that actually works.

6 types of employee recognition programs—and what they cost 

Most organizations don’t run one recognition program. They run several, layered together. The mix depends on team size, culture, and goals—but the programs that move the needle tend to pair recognition with rewards. Quantum Workplace research shows 82% of employees say recognition is more impactful when it includes a reward, not just an acknowledgment.

Not sure what types of recognition employees actually want? The answer might surprise you. Here’s how the most common program types break down:

Peer-to-peer and values-based programs cost relatively little per instance and work best at high frequency. Milestone and performance-based rewards are easier to predict and plan for in advance.

Programs that run only one type—usually manager-led—tend to create recognition deserts. Some teams feel celebrated every week. Others go months without acknowledgment. That inconsistency shows up in turnover data long before it shows up in exit interviews.

What a well-built recognition program looks like 

A budget funds a program. But the program has to be designed well for the budget to do anything. The organizations that see recognition actually change their culture share five traits.

• It focuses on what matters most. Recognition should highlight specific behaviors and outcomes—not generic praise. Tying recognition to company values and meaningful contributions ensures the program reinforces performance, not just presence.
• It feels meaningful and specific to the people receiving it. One-size-fits-all recognition rarely lands. Employees should have choice in how they’re rewarded, messages should be personal, and the program should feel relevant to individuals—not like a system checking a box.
• It happens easily and frequently in the flow of work. Recognition works best when there’s no friction. That means integrating into tools like Slack and Teams, refreshing points monthly so the habit stays active, and trusting employees to recognize great work in real time without approvals slowing things down.
• It scales visibly across the organization. When only managers can recognize, most great work goes unseen. Opening recognition to every employee—and making it public—turns individual moments into shared culture. Visibility is what transforms recognition from an occasional gesture into a daily norm.
• It serves as a real-time performance signal for leaders. When recognition is frequent and tied to meaningful work, patterns emerge. Leaders can see who is driving results, where contributions are going unnoticed, and how recognition connects to broader performance trends. Recognition stops being just appreciation—it becomes intelligence.

How to allocate the employee recognition budget effectively 

Getting the budget approved is one conversation. Deciding how to split it is another. Most programs divide spend across three buckets: everyday recognition, milestone recognition, and performance-based awards.

A rough starting point: weight the largest share toward everyday recognition. Research shows employees who receive recognition monthly or more are 80% more likely to be highly engaged. Frequent and small beats rare and large, almost every time. is worth a read before you build your proposal.

A few allocation decisions have outsized impact on whether the budget actually performs:

• Give managers a recurring allocation, not a one-time pool. A manager who runs out of budget in March will stop recognizing in April. Monthly or quarterly replenishment keeps the program running year-round. The monthly refresh also encourages frequency—managers spend what they have because unused points don’t carry over.
• Open peer-to-peer to everyone, with a modest per-person allowance. Peer recognition scales across teams in ways no single manager can cover alone. It also makes great work highly visible—and that visibility creates richer signal for leaders trying to understand who’s driving impact and where.
• Reserve a portion for rewards choice. Employees who have the opportunity to choose what they redeem their rewards for are 87% more likely to say recognition feels meaningful, versus 52% when they don’t. A flexible rewards catalog doesn’t require a larger budget—it just requires optionality.
• Budget separately for milestones. Work anniversaries, new hires, and promotions are predictable enough to plan for. Keeping a separate allocation means managers aren’t pulling from the everyday pool to cover them.

Checklist: Is your recognition program ready to budget? 

Before submitting a budget proposal, make sure you can answer these questions:

• Do you know your current employee and manager headcount?
• Have you defined which recognition types you want to fund — peer, manager-led, and formal milestones?
• Do you have a per-employee-per-month target or range in mind?
• Can you quantify what turnover costs your organization?
• Is recognition currently consistent across teams, or reactive and manager-dependent?
• Do you have a way to track recognition participation and redemption over time?
  

If several of these are still “no,” that’s a signal you need a starting point—not more benchmarks.

How to make the case internally 

Getting a recognition budget approved is a different problem than knowing what to spend. The conversation with finance or a C-suite leader goes better when you bring three things:

• A specific number. Not a range. A number with a per-employee breakdown that reflects your actual headcount and program design.
• The retention math. Frame the spend relative to turnover cost. If your average employee salary is $65,000 and replacement costs 75% of that, one retained employee justifies a meaningful recognition investment.
• A phased approach. Showing good/better/best options—starting at $5 per employee per month and scaling from there—takes the all-or-nothing pressure off the conversation and makes it easier to get started.

The real cost of skipping this conversation 

Lack of recognition is a top three reason employees leave their jobs. Not compensation. Not workload. Recognition—or the absence of it.

That context matters when you’re making the case to leadership. Recognition isn’t a perk line item. It’s a retention strategy—and retention has a dollar figure finance already understands.

Replacing an employee typically costs 50–200% of their annual salary, depending on role and seniority. A recognition program at $5–$10 per employee per month is a fraction of one employee’s departure.

Build your recognition budget — free, in five minutes 

Recognition programs don’t fail because HR leaders don’t care. They stall because nobody hands them a number they can actually use.

Quantum Workplace’s free Employee Recognition Budget Calculator takes about five minutes. You enter your headcount, design your program, and walk away with a personalized annual budget, a per-employee breakdown, and a summary already formatted to share with leadership or finance.

It’s built for HR leaders who are done guessing—and ready to build something real.

Build My Recognition Budget →

How much should an employee recognition budget be per employee?

Meaningful programs start around $5 per employee per month. Top-performing organizations often spend $50–$60 per person per month when including rewards. The right number depends on your program design, company size, and goals.

What’s the difference between recognition and rewards?

Recognition is the act of acknowledging an employee’s contribution. Rewards are the tangible benefits employees receive—gift cards, experiences, or redeemable points. Programs that combine both are significantly more impactful than acknowledgment alone.

How do I build a recognition budget for a 500-person company?

Start with your headcount and decide which recognition types you want to fund. A peer-to-peer program at $5/employee/month for 500 employees runs $30,000 annually. Layer in awards and rewards based on your goals and build from there.

What should a complete employee recognition program include?

Peer-to-peer recognition, manager-led awards, redeemable rewards, formal milestones, and a platform that makes recognition consistent and trackable across the organization.

How do I justify a recognition budget to leadership?

Ground the conversation in turnover cost. Recognition programs cost a fraction of replacing one employee. Pair that with a specific number — not a benchmark—and bring a phased investment plan they can say yes to incrementally.

Is recognition software worth it for smaller teams?

For organizations with more than 100 employees, yes. Without a platform, recognition stays manager-dependent and inconsistent. A platform makes recognition part of the daily workflow and gives HR the data to measure what’s actually working.

At the recent AI and Society Forum at MIT, experts from across the Institute discussed the potential benefits and dangers of technological innovation on labor, the nature of work, civil discourse, election administration, and other topics.

The event featured individual research presentations and panel discussions, as well as a musical performance exploring the use of generative artificial intelligence in the arts.

The forum was co-organized by the School of Humanities, Arts, and Social Sciences (SHASS) and the Social and Ethical Responsibilities of Computing (SERC). It was presented in collaboration with two of MIT’s strategic initiatives: the MIT Generative AI Impact Consortium (MGAIC) and the MIT Human Insight Collaborative (MITHIC).

Agustín Rayo, the Kenan Sahin Dean of SHASS, and Dan Huttenlocher, dean of the MIT Schwarzman College of Computing, provided opening remarks.

Rayo said bringing scholars from across MIT together was intentional because understanding AI’s impact requires expertise from disciplines throughout the Institute.

“Paying attention to the societal consequences of AI is not a departure from MIT’s mission; it’s a way of ensuring that our technical leadership has maximum impact,” Rayo said.

Huttenlocher added that computing and AI’s rapid growth makes it critical to support interdisciplinary conversations and research.

“Understanding where AI excels and where it falls short is essential not only to unlocking its benefits, but also to avoiding critical errors, overreliance, and unintended consequences,” Huttenlocher said.

Jobs and AI 

Held in the Tull Concert Hall in MIT’s Linde Music Building, the May 12 forum opened with a keynote presentation from economist David Autor, the Daniel (1972) and Gail Rubinfeld Professor in the MIT Department of Economics. Autor challenged the common narrative that AI will simply eliminate jobs by proposing instead that technology’s impact depends on how it affects the scarcity and value of human expertise. 

“When I think about how technology interacts with the value of labor, I think about it in terms of how it changes the scarcity of expertise, whether it makes it more valuable or whether it makes it more of a commodity,” he said.

Autor said that what matters is whether automation removes routine supporting tasks or removes expert tasks. He argued that AI will likely create new specialized work, requiring proactive policies around worker training, wage insurance, and broader capital ownership.

A panel discussion followed, featuring experts from MIT discussing how work is changing and what it means for society. 

Daniela Rus, the MIT Panasonic Professor of Computer Science and director of the Computer Science and Artificial Intelligence Laboratory (CSAIL), described excitement around ways AI could enhance the workplace.

“I’d like to imagine the robot as your friend and assistant, as someone who watches you and figures out how to help you as someone you can task at a high level,” she said. 

Still, Rus said, human judgment remains critical in decision-making.

“We could really think about co-work with the AI tools, but the role of the human as the decider, as the person with good judgment, as the person deciding the next step, whatever that is, remains super important,” she said.

David Mindell, professor of Aeronautics and Astronautics and the Dibner Professor of the History of Engineering and Manufacturing in the Program in Science, Technology, and Society, says the nature of work has constantly changed over the years, but “what matters is the new work.” 

“We need to be supporting individuals, the economy, professions, to constantly be creating the new work,” he said. “It’s absolutely imperative that we give the tools to the young people and let them do what they find creative and show us what the new work is going to be.”

Panelists also talked about the need to maintain safety standards, while also exploring ways to find efficiencies. Mindell used an example of cargo flights that require six pilots due to the length of the flight.

“We don’t know how to take that six number down to five yet, much less two, one, or zero. There’s a lot of money behind solving that problem, but there’s also a very rich system that has evolved to make those systems safe,” he said.

Sendhil Mullainathan, the Peter de Florez Professor with dual appointments in the MIT departments of Economics and Electrical Engineering and Computer Science (EECS), described a vision of AI’s utility and growth that offers productivity improvements, but also cautioned, “I think it’s very much worth differentiating productivity gains from things that actually drive long-term growth.”

Either way, Mullainathan said, it’s clear we’re entering a time of high variance with regard to AI’s impact on the workforce.

“If you said, ‘exactly how will organizations restructure?’ I don’t know. But is there going to be a lot of restructuring? It’s hard to believe there isn’t going to be a lot of restructuring. And in some sense, if we know that what we’re entering is a period of high variance, that itself is incredibly informative,” he said.

Democracy and AI

The day’s second session focused on AI technology and its impact on democracy. 

Chara Podimata, the Class of 1942 Career Development Assistant Professor and assistant professor of operations research and statistics in the MIT Sloan School of Management, presented her research on auditing large language models for bias in election information.

“Algorithms decide a lot of things about our lives right now,” she said. “With regard to chatbots and election information, if I take two people and they interact with the same chatbot … how will the chatbot respond? How will it personalize the information it gives to these people?”

A longitudinal study of 12 major models during the 2024 U.S. presidential election season found responses varied dramatically based on stated demographics and political leanings. Her research team is now working on a new audit of the 2026 U.S. midterm elections, using a redesigned survey with input from political science experts.

During a panel discussion, experts raised concern about the potential for AI to erode democratic norms and processes, but also explored potential positive outcomes.

Bailey Flanigan, the Theodore T. Miller (1922) Career Development Professor in the Department of Political Science, who holds an MIT Schwarzman College of Computing shared position with EECS, said she’s skeptical of how some are applying AI as a tool that can get people to reach decisions or consensus more quickly.

“And there is a reason to think that this is nice because it is more efficient. It’s easier. But it loses a lot of these procedural elements of democracy that are the rituals of how we come together and make decisions,” she said. “And I think it’s a mistake to forget about that when we start thinking about automation.”

Charles Stewart III, the Kenan Sahin (1963) Distinguished Professor of Political Science and founding director of the MIT Election Data and Science Lab, said one challenge is that governmental structures do not evolve at the same rate as technology.

Stewart said his biggest concern is the potential for AI to lead to chaos during and after elections.

“If and when things go wrong, they can go really bad, and really wrong. If an election is called into question, that can lead to violence,” Stewart said.

“We’ve already seen in the low-tech eras election results being manipulated. What worries me is what I’m going to observe this coming Election Day, and the Wednesday after, and if AI has helped to create irreversible disruptions to the election system,” he added.

Lily Tsai, the Ford Professor of Political Science and director and founder of the MIT Governance Lab (MIT GOV/LAB), said in many ways, AI runs against the democratic norms and commitments necessary for a healthy democracy.

“It is really important not just in terms of design principles, but the commitments of designers to be familiar with the values and principles that characterize what democracy is based on: agency, political equality, mutual respect, inclusion, and autonomy,” Tsai said.

Tsai also noted her research has shown some people are more comfortable interacting with machines. She described a “Socratic dialogue chatbot” her team designed that asks people to articulate the thinking behind their beliefs and positions.

“And that actually, interestingly, seems to moderate their policy position in the process,” Tsai said. “So there are absolutely examples of ways in which AI can have positive impacts on democracy. But it really is about designing with the right principles and evaluating them rigorously.”

Upcoming Changes to the SmartThings API 

Over the last several years, the SmartThings platform has grown into one of the most robust orchestration layers in the smart home industry, with over 460M+ registered users and hundreds of WWST partner brands — and our API is the gateway that connects third party apps and platforms to the SmartThings ecosystem. 

Today, our commercial partners use it to power everything from short-term rental platforms and energy management systems to API aggregators and custom smart home dashboards, while our developer community uses our API to optimize their personal smart homes.

To ensure the SmartThings API remains capable, reliable, and secure at this scale, we are evolving our infrastructure. In the upcoming months, we will introduce dedicated, paid commercial API tiers as well as a $4.99 a month plan for non-commercial, individual developers.

To note, this update does not affect the millions of SmartThings users who use the SmartThings App with thousands of WWST partner devices.  

Why are we evolving the SmartThings API? 

As our commercial partners and developer community continue to build integrations, they are asking and expecting more from our infrastructure. To support the next generation of smart home innovation at scale, we are transitioning to a structured API model. This evolution allows us to invest heavily in the enterprise-grade features our partners and users have been asking for:

• Platform Stability: Upgrading our platform to enhance our enterprise-grade reliability required for high-volume, commercial deployments.
• Optimized Integrations: Building upon our infrastructure to further support scalable integrations.  
• Expanded Capabilities: Continuing access to Samsung devices such as TVs, smart refrigerators, and more with device states, controls, and health parameters as well as future functionality to build richer integrations.

Powering the Next Generation of Smart Home Services: 

We’re building on the SmartThings platform, shaped by your feedback, to deliver even more powerful tools for creating premium IoT experiences. The SmartThings API offers different tiers that can support a high rate limit, gives access to our multi-protocol ecosystem (Matter, Cloud-to-Cloud, Zigbee, Z-Wave), and contains deep Samsung integration required to scale your business on our backend and power a variety of use cases, including:

• Short-Term Rentals, Hospitality, and Property Tech: Automate access control, optimize climate, and enable seamless property management across thousands of units at scale.
• Telecommunications: Enhance broadband and 5G subscriber apps with premium family care and security solutions.
• Security: Power smart alarms, access systems, and remote monitoring with our ecosystem of devices and consistent, real-time API data like device states.
• Senior Care: Enable non-intrusive wellness tracking, activity insights, and automated routines to safely support independent living and aging in place.
• Family Care: Deliver peace of mind with real-time sensor data, keeping families connected and homes running smoothly.
• Insurance: Leverage real-time home state data and predictive analytics to proactively prevent damage, reduce risk, and lower claim costs.
• Energy Management: Build advanced tracking algorithms and cost-saving utility dashboards.
• With many more use cases and verticals. 

New Tools: 

• Upgraded Developer Experience: We will deliver a new Developer Center experience and refreshed documentation for API users to make the API as easy to use and implement as possible.
• The API Usage Dashboard: In the new Developer Center, we will be launching a new API Dashboard. This dashboard will allow partners and individual developers to track API call volume with flexible time-series data. The dashboard will give you current usage and data points to optimize your code and determine which pricing tier fits your needs.

These updates will make onboarding faster and API management more intuitive. 

Next Steps & Timeline: We are targeting availability for the new commercial tiers and personal plan offering in October 2026 – stay tuned for additional details. 

Please note: Your current API usage will not be interrupted today. We are announcing this now to give our API users ample time to prepare for the transition.  Free access will remain available through Q3. We will not begin applying the new usage limits or phasing out free access until October 2026.

• For Current Commercial Partners: Please reach out to us at partners@smartthings.com to ensure a seamless transition.  
• For Current Personal Users: We will share more information detailing the full timeline, transition guidelines, and paid plan. 
• New to the SmartThings API? Are you evaluating SmartThings as the backend for your business? We are currently onboarding new partners to discuss SmartThings API capabilities, pricing, and scalability. Please fill out our form to learn more.

Thank you for building with SmartThings. Our investment in our API and transition to a paid model are efforts to provide an incredible service that will continue to support your needs.  

— The Samsung SmartThings Team

A new chip developed by MIT researchers could help tiny, low-power UAVs avoid obstacles as they zip around tight corners inside an industrial HVAC system to check for gas leaks.

The chip allows small autonomous robots and other battery-limited devices to construct detailed 3D maps of their environments in real-time using only about as much power as a single LED. A robot could use such a map to plan a collision-free path to reach its goal.

Typically, generating such thorough maps requires power-hungry systems and a great deal of memory to build and store 3D representations of the obstacles in a robot’s environment.

The MIT researchers took a different approach by combining an extremely efficient mapping algorithm with specialized hardware designed to accelerate its workload, which minimizes memory and power consumption. 

This system-on-a-chip consumes only about 6 milliwatts of power, a fraction of the power required by other systems. 

This low-power operation could also make the chip well-suited for lightweight augmented reality headsets that can be worn for extended periods, for applications like educational medical simulation or detailed repair and assembly work.

“This paper showcases a key example of how you can leverage co-design of the algorithm and hardware to really push energy efficiency. While there has been a lot of work looking into compact 3D maps, what stands out about this work is that it also ensures that the process to generate those maps is as efficient as possible. Our chip allows you to store very large maps in a very small space, and do it in a very energy efficient manner,” says Vivienne Sze, a professor in the Department of Electrical Engineering and Computer Science (EECS), a member of the Research Laboratory of Electronics (RLE), and senior author of a paper on the chip.

She is joined on the paper by co-lead authors and MIT graduate students Zih-Sing Fu and Peter Zhi Xuan Li as well as Sertac Karaman, a professor of aeronautics and astronautics and the director of LIDS. The work was recently presented at the IEEE Very Large-Scale Integrated Circuits Symposium.

A more compact map

For a robot, generating a 3D map that includes the obstacles in its environment usually demands a lot of power because it must store images captured by its camera, and process all the 3D pixels in each image multiple times.

Instead of representing the environment using 3D pixels, which are cubes called voxels, the MIT researchers utilized a technique that maps the obstacles in space using ellipsoid blobs called Gaussians. 

The size, shape, and thickness of these ellipsoids can be smoothly adapted, so they match the shape of curved objects more efficiently than if one uses rigid, cube-shaped voxels. 

Importantly, the map captures the obstacles and free space around the robot, and together these let the robot plan a safe, collision-free path. Mapping obstacles and free space with voxels typically consumes a lot of memory, which makes traditional methods power-hungry. Because Gaussians can flexibly fit the geometry, a single elongated ellipsoid can represent a region that would take many voxels, so occupied surfaces and free space are captured far more compactly.

For their new system-on-a-chip, called Gleanmer, the researchers employed an algorithm their lab developed called GMMap that efficiently generates a 3D map of the robot’s environment using Gaussians to represent obstacles. 

With traditional approaches, a robot would need to load and process each depth image several times to adjust the size and shape of the ellipsoids. The system would usually construct Gaussians by comparing all the pixels in an image to each other. But the amount of memory and power needed to do this remains too high for many edge devices.

To solve this problem, the MIT researchers invented a technique that can generate highly accurate Gaussians from depth images with only one pass, after which they can discard the images, so the chip never has to store an entire image at once. 

Instead of comparing each pixel to every other pixel in the 3D image, their algorithm assumes that nearby pixels belong in the same Gaussian, so it only needs to compare each pixel to its neighbors.

“At any point in time, we only need to store a few pixels in memory, which significantly reduces the memory footprint our algorithm requires,” Li says.

Leveraging co-design

But as the robot moves through the space, it usually sees the same object from different viewpoints. When it generates Gaussians, some will overlap because they represent the same object. This can make the 3D map too large to store on an edge device.

Fusing overlapping Gaussians makes the map more compact, but doing so typically requires the algorithm to process many raw pixels stored in memory. The researchers developed a novel technique to perform this fusion process directly on overlapping Gaussians, without needing to revisit the original pixels. Since Gaussians are more compact than pixels, this significantly reduces memory and power requirements.

The same principle runs through their algorithm — most computations operate directly on compact Gaussians rather than the original pixels, enabling energy efficiency.

The researchers exploit this principle to design a chip that keeps the Gaussians it is actively working on within small, fast on-chip memory right beside the computational units. This is only possible because the Gaussian map is so compact.

The Gaussians the robot needs to work on next are waiting in the on-chip memory units, so they don’t need to be fetched from more distant, power-hungry, off-chip storage. 

“By having a dedicated memory that just stores the objects you’ve seen in the previous few frames, you can access the data much more efficiently,” Fu explains.

They tested the system-on-a-chip by reconstructing a range of diverse, pre-existing 3D environments. The chip can also reconstruct obstacles and free space directly from live data streamed from an iPhone camera.

Gleanmer generated detailed 3D maps in real-time while consuming about 6 milliwatts of power. It required only about 2.5 percent of the power that the best existing chip for map construction would need. 

By reusing compact Gaussians along the path as it plans, the chip lets a robot chart a safe trajectory using only about 20 percent of the energy it would otherwise need.

“We reduce the memory consumption by making sure the algorithm is efficient. Then we accelerate the workload that is performed by that efficient algorithm, so in the end, our chip is as efficient as possible,” Li says.

The researchers plan to further improve energy efficiency by moving the processing units on the chip closer to the sensors that gather environmental data. They could also explore additional applications, such as the use of Gaussians to represent schematics. This could help AI systems reason about complex blueprints more efficiently.

“Real-time 3D mapping has been the missing piece for small autonomous systems. A drone inspecting a pipeline or a pair of AR glasses navigating a room both need to understand the space around them — instantly, continuously, and at almost no power cost. Gleanmer makes that possible for the first time in a chip you can hold between your fingers,” says Karaman.

This work is supported, in part, by the MIT-MathWorks Fellowship, Amazon, the U.S. National Science Foundation, and Intel. 

An auto factory worker can remember the storage bin where she left a partly assembled component the night before, and quickly return to that spot to pick it up. But robots that may work side-by-side with her would struggle to develop and access this same type of “spatiotemporal” memory.

Now, MIT researchers have developed a long-term memory framework that allows robots to rapidly form and recall a detailed mental model of complicated, large-scale environments.

In the future, this advance could allow the factory worker to send a robotic assistant to fetch the item, simply by asking it to “go and grab the component we started assembling last night.”

This new method combines advanced map representations with rich descriptions of the environment that the robot gathers as it travels over a long period of time. The robot can quickly access this memory to answer complex queries about its environment in plain language.

This memory framework, which answers questions more accurately than state-of-the-art methods, runs fast enough for a mobile robot to use in real-time.

In addition to its potential uses in robotics, this method could have applications in augmented reality systems that aid maintenance workers in anomaly detection or assist commuters in wayfinding.

“If we want robots to work side-by-side with humans and interact better with humans, they must speak the same language. The robot must be able to reason about time and space the same way humans do. That is essentially what our method is doing. It is turning a traditional map into a language-based map that is easier for the robot to think about and access using language,” says Luca Carlone, an associate professor in MIT’s Department of Aeronautics and Astronautics (AeroAstro), principal investigator in the Laboratory for Information and Decision Systems (LIDS), and director of the MIT SPARK Laboratory.

He is joined on the paper by lead author Nicolas Gorlo, an MIT graduate student; and Lukas Schmid, a former research scientist at MIT and now professor at the University of Technology Nuremberg in Germany. The research was recently presented at the Conference on Computer Vision and Pattern Recognition (CVPR).

Spatiotemporal memory

Memory allows an artificial intelligence system, like a chatbot, to answer complex questions and reason about previous interactions with its user.

“We want to design a new type of memory, a spatiotemporal memory, that enables an AI-powered robot to remember real interactions and sensor observations. Like ChatGPT, but grounded in the real world and capable of answering any question about the environment, like ‘Where did I leave my wallet?’” Carlone says.

To develop such a memory framework, the MIT researchers bridged two lines of work: computer vision and robotic mapping.

Multimodal computer vision models can understand and richly describe the objects in a scene, but they often only process a single annotation at a time. On the other hand, robotic mapping frameworks create 3D maps of an environment, like an entire apartment or university campus, but usually lack detailed descriptions of objects or are computationally expensive.

The method the MIT researchers created, called Describe Anything, Anywhere, Anytime, at Any Moment (DAAAM), takes the best of both approaches.

Using DAAAM, as a robot traverses its environment, it attaches rich descriptions to objects it sees. For instance, the robot may note that a particular building on the MIT campus is called the Stata Center and is designed with a certain type of architecture, or that a bike rack holds five bicycles and the red one has a flat tire. 

It stores this detailed information in a 3D map-based representation that is arranged spatially, so objects will be grouped into separate regions. In this way, the robot can remember that the red bicycle with the flat tire is in the bike rack outside the Stata Center.

But existing techniques that capture such rich descriptions typically take a few seconds to annotate a few objects. This is too slow for real-time performance, since a robot might see hundreds of objects during a few minutes of exploration.

“The faster the robot can form this spatial memory, the more efficient it will be performing actions in the environment,” Carlone adds.

Streamlining the process

To speed things up, DAAAM aggregates nearby objects as it travels and uses an optimization method to select key frames to annotate. These are images with the clearest view of multiple objects, allowing the system to thoroughly describe several items in parallel, speeding up computation tenfold.

As the robot explores the space, it attaches each batch of annotations to multiple objects in a particular location on the 3D map.

“We annotate every object only once, so our framework can run in very large-scale environments in real time. And by clustering objects into regions, it can answer a wide range of queries about objects and locations in the environment,” Gorlo explains.

Once the system builds this spatial memory, it must retrieve information from an enormous database of objects and descriptions in an efficient manner. 

To enable this, the researchers used an LLM that calls on various tools, which can quickly retrieve specific information in a way that reduces hallucinations. This allows DAAAM to answer a user query accurately in only a few seconds. 

For instance, if one asks a robot about a certain sculpture it saw near an MIT campus building, DAAAM can use a semantic search tool to retrieve information based on the word “sculpture” or a different tool to retrieve information based on the location of the building.

When tested and compared with other methods, DAAAM was between 21 percent and 53 percent more accurate, depending on the question type. 

In the future, the researchers want to expand DAAAM so the system can capture significant events that happened in the environment. They are also working to incorporate confidence levels into the system’s responses.

“Ultimately, we want to have robots that can help with any sort of tasks. With this framework, we are trying to create the foundations to enable a generalist agent that can do anything you ask,” Gorlo says.

This research was funded, in part, by the U.S. Army Research Laboratory and the Office of Naval Research. Carlone is currently on sabbatical as an Amazon Scholar; this article describes work performed at MIT and is not associated with Amazon.