Humanoid robotics company 1X has unveiled a new tendon-driven robotic hand for its NEO humanoid platform, saying the design brings near human-level dexterity, strength and tactile sensing to the robot while enabling more advanced AI-driven manipulation capabilities. The company says the new hand features 25 degrees of freedom (DoF), including 22 fully actuated joints in […]
How AI agents are transforming industrial operations beyond manufacturing
Please switch off your ad blocker.
Our website relies on ads to provide free content and sustain our operations. By turning off your ad blocker, you help support us and ensure we can continue offering valuable content without any cost to you.
We truly appreciate your understanding and support. Thank you for considering disabling your ad blocker for this website
Dismiss
Monumental raises $32 million to expand autonomous construction robots
Monumental, the Amsterdam-based tech company automating construction with robotics and software, has announced a $32 million Series B led by Khosla Ventures, with participation from existing investors Plural and Hummingbird. The funding will grow Monumental’s world-class team of hardware and software engineers, scale the number of robots it can deploy across Europe and the UK, […]
Logistics firms, robotics startups and brands set up shop in Industry City, fueling a retail tech hub in Brooklyn
E-commerce is one of the fastest-growing retail subsectors in New York City, with new consumer brands launching at a relentless pace and 2.5 million packages delivered across the city each day. Now, the companies that manufacture, print, fulfill and automate for these brands are racing for space in the country’s most competitive real estate market. […]
Enterprise AI Agents Are Taking Over – Is Your Infrastructure Built to Last?
Enterprise AI agents are going from labs to corporate systems. They can track workflows, generate reports, retrieve data, coordinate tasks, and make decisions across applications. This increase allows faster operations and better service, but it also strains the technical environment. AI agent infrastructure pillars that can handle changing workloads, secure access, maintain data quality, handle […]
How AI agents are transforming go-to-market operations for robotics companies
RevOps for robotics OEMs brings sales, marketing, and customer success teams under one strategic umbrella.
These teams must innovate ways to attract and retain customers, increase average order value (AOV), and maximize customer lifetime value (CLV) to grow revenue, all while optimizing costs.
Therefore, efficiency is crucial, making tech stack management one of the most critical roles of RevOps.
Go-to-Market (GTM) AI
These stacks are evolving to include Go-to-Market (GTM) AI agents that work with CRMs and ERPs for improved predictive forecasting, market data analysis, buyer intent tracking, and conversation intelligence, as agentic workflows handle the time-consuming work of lead routing, scheduling, and even personalized outreach.
The result is a unified RevOps team with more freedom to innovate creative strategies and intervene with precision to grow and preserve revenue.
Building an Agent-Native GTM Tech Stack
RevOps engineers use GTM AI APIs to build core tech stack layers, like conversational intelligence layers that transcribe and analyze prospect calls, meeting notes, and emails. GTM agents identify proactive and effective sales tactics that are working while flagging stalled conversations that could use a new approach.
AI Agents are being integrated into intent layers to monitor high buying intent signals, such as:
- The hiring of new robotics OEM CEOs
- Company expansion announcements
- Whitepaper downloads
- Workforce hiring surges
- Funding announcements
- Product launches from prospects’ competitors
To enrich prospect profiles, tech stack agents crawl public data sources, news feeds, and professional robotics networks to build highly targeted contact lists with more context. This information is visualized as contact graphs with relevant points of contact.
The orchestration layer essentially “ingests” the data pulled and analyzed from previous layers, assigning broader GTM tasks to other agents, like personalized outreach emails to purchasing managers.
GTM Agent Workflows in Action
To visualize these agent-powered stacks in action, imagine a plausible scenario in the robotics OEM sector, like a senior engineer from a major automotive manufacturing company viewing a datasheet for a particular robotic arm product.
Immediately, the stack’s routing AI agent “asks” the internal CRM system if an active regional OEM partner owns that territory.
A data enrichment agent then pulls the company’s recent manufacturing expansions, funding announcements, and current job postings to identify exact technical pain points based on the company’s current manufacturing technology stack.
There is now enough context for the next AI agent to draft a hyper-personalized outreach email to the prospect, speaking directly to their pain points, while referencing the robotics datasheet that the engineer viewed. The agent includes additional insight into how the product aligns with the prospect’s broader operational goals.
When the prospect books a meeting with the sales team, it triggers another stack agent to check the ERP system for current stock availability, delivery timelines, and active distributor agreements, logging a verified sales opportunity in the CRM for the sales reps.
Optimize Your Tech Stack With AI Agents
Think of GTM AI agents as an extension of your RevOps team. Calculate your revenue potential and cost savings from agent-powered conversational intelligence, data enrichment, and orchestration. Consider working scenarios for GTM AI and the impact of deep context on conversion rates.
Keep your stack’s news feed layer up to date with the latest headlines in robotics and automation.
Main image: Simon Kadula, Unsplash
Related stories you might also like…
Can AI build a jet engine? JARVIS Challenge tests role of AI copilots in tough-tech engineering | MIT News
Artificial intelligence has rapidly transformed software engineering. Generative AI and large language models (LLMs) can create huge volumes of code and documentation; machine-learning algorithms can monitor performance and detect security vulnerabilities. But when the task is to conceive, design, and make a complex physical system such as a jet engine, are those AI tools equally transformative?
This past semester, the JARVIS Challenge (Jet-engine AI Research and Validation Intensive Sprint) set out to explore whether AI can compress the design-build-test cycle, asking MIT undergraduates to discover whether AI can help them to build faster and better.
“The JARVIS challenge showed that AI can substantially accelerate safety-critical hardware engineering, but engineering judgment remains the decisive differentiator. An AI-native engineer is not defined by using AI, but by leading it — knowing when to trust it, when to challenge it, and how to translate AI outputs into working hardware. Manufacturing — not engineering design or analysis — remained the fundamental rate-limiting step,” says Professor Zolti Spakovszky, director of the MIT Gas Turbine Laboratory.
The teams, the tools, the task
The challenge gave undergraduates four weeks to design, fabricate, assemble, and test a small gas turbine aero engine, using AI as their primary engineering partner. The objective: build a “JARVIS-class” single-spool jet engine producing 50–100 pounds of thrust, running on Jet-A, and completing five 60-second runs. Teams had total freedom over design, materials, and fabrication.
Representing nearly every department in the School of Engineering, 31 students organized into seven teams, ranging from all first-years to senior-heavy groups. Many of the competitors initially had little experience in turbomachinery, compressible flows, or, in the case of the younger students, even thermodynamics. Many had never seen the inside of a gas turbine before signing up to build one.
At their disposal: MIT’s machine shops and manufacturing vendors; commercial software including Concepts NREC, SolidWorks, and ABAQUS; and various test rigs for characterizing and assembling individual components.
The teams also had access to MIT Parley, a newly launched platform that aggregates frontier large language models through a single interface. Through Parley, JARVIS leads could see directly how the students were using the AI tools, including their prompts, the cost per prompt, the specific LLMs being used, and other critical information. The JARVIS leads secured early access to Parley for all participants, and with financial support from MIT Lincoln Laboratory, the Department of Mechanical Engineering, and corporate sponsors Safran, Voyager Technologies, and Beehive Industries, students had access to essentially unlimited use of AI.
The sponsors were drawn by recruiting interest and genuine curiosity about how AI might reshape engineering workflows.
“We see this as the future of engineering,” Ryan (Hal) Hefron of Voyager Technologies told the students. “You’re honing skills that are not just nice to have — they’re going to be the future baseline in the engineering workforce.”
Vincent Garnier, managing director of Safran Tech, watched the competition unfold with excitement. “JARVIS was a genuine experiment, a learning endeavor. We frankly didn’t know what to expect, from the students or from the AI models. What struck me coming from the students was: first, the enthusiasm to explore; then, as the project developed, they all came to the cool-headed realization of what AI could or could not help them with, and then almost instantly adapted for that,” he says. “It makes me confident that this generation of leading engineers will probably not fall prey to easy and shortsighted use of AI, and will do so by keeping ever more in contact with experiments — physical or thought experiments.”
The faculty leadership — professors Zachary Cordero, Zolti Spakovszky, Masha Folk, and Andreea Bobu of the Department of Aeronautics and Astronautics, along with Lincoln Laboratory engineers and a team of teaching assistants — were there to ensure safety. In weekly progress reviews, they would critically evaluate the student progress and assess how the students were using AI.
Spakovszky developed a careful technique for guiding teams in the right direction without giving away answers or providing help. After a team’s presentation, he might ask: “Do you know what a rabbet fit is? Take in the comment.”
Where AI helps and hurts
By the end of week 1, one team withdrew from the competition; the others had, with varying degrees of success, developed an initial design for their gas turbines. Different teams used AI to summarize textbooks, teach them to use design software, source vendors, create Excel sheets, answer specific questions, find references, and create comparative analysis between design decisions. One team created an agent in Parley and tasked it with serving as their project manager.
By week 2, teams had to start working on detailed CAD designs, ordering parts, and prototyping their combustors. This is where the teams started to hit limitations in their use of AI. While Claude and ChatGPT were good at offering design alternatives and filling knowledge gaps, teams found that the hallucinations, sycophancy, and lack of physical understanding that have become notorious features of generative AI were undermining their confidence and slowing them down.
“AI is a helpful tool, great at finding information, helping organize things, and can write well, but it can’t do design,” says Elizabeth Tupaj, a member of team 811 Crew. “The moment the engineer doesn’t know what is going on and the AI is in charge is the moment the design becomes unreliable, at least with AI at its present capabilities.”
Teaching assistant John Zhang notes, “seeing this firsthand with the students reminded me how much first impressions matter. If the students couldn’t get answers from the AI early on, they quickly grew frustrated and formed a lasting opinion that precluded them from using it later.”
In the final weeks, the finalists hit another obstacle no AI could solve: working with vendors. “AI searches found vendors we had no rapport with, who had no interest in our tight timeline,” students reported. “The vendors who came through were the ones our team had personal relationships with.”
Of the three finalists, only Fast and Fractured achieved first-attempt ignition of their mini-combustor. The team had used AI heavily for trade studies and architecture comparisons, arriving at a viable design despite none of them having prior gas turbine experience.
“The JARVIS Challenge showed what’s possible when you combine AI-enabled design with motivated students and a culture of rapid experimentation,” says Masha Folk, the Charles Stark Draper Career Development Professor of Aeronautics and Astronautics. “The moment that stood out most was when the first student-designed combustor was installed on the test stand. It ignited flawlessly, ramped to full power, transitioned to dual-fuel operation, and then sustained stable combustion on 100 percent Jet-A fuel. This was proof that we can dramatically accelerate the cycle of design, build, and test while giving students hands-on experience with a real engineering challenge.”
At the vanguard of AI-native engineering
By the end of May, the two more senior teams – Fast and Fractured and 811 Crew – had completed full engine tests. Fast and Fractured, with their AI-assisted design, were delayed by vendor headaches week after week, but finally made it to test. Unfortunately, their hot fire was cut short when the rotor rubbed and seized against the stationary housing. Team 811 Crew, however, who had more exposure to turbomachinery and propulsion concepts going into the competition, emerged victorious. Their engine started, successfully transitioned to Jet-A, and generated net thrust.
“As we stood there with the air-starter, hearing their engines spool up and watching them spit fire, it felt like my heart was racing out of my chest. There were so many ways it could go wrong! What these students accomplished in such a short time span is nothing short of amazing,” says PhD student Joe Chiapperi.
The 811 team had been resistant to using AI throughout the competition, trusting instead to their fundamentals and teamwork. “We had people who were at least somewhat familiar with the design software, mechanical engineers who knew how to build anything, and aerospace engineers who had taken classes on the design of gas turbine engines specifically,” says Tupaj.
From the start of the JARVIS Challenge, younger students used Parley more frequently and cleverly, while the juniors and seniors leveraged deeper experience.
“JARVIS taught me that getting value from AI takes two things: enough expertise to judge what it tells you and catch it when it’s wrong, and enough curiosity to actually lean on it where it could help,” says Professor Andreea Bobu. “The team that moved fastest in the sprint was experienced and leaned heavily on AI to get there. The team that eventually won was more resistant to AI; they had the expertise, but that skepticism made them slower. The sweet spot seems to be knowing enough to stay in charge of the tool, and being eager enough to pick it up in the first place. To me, that’s the real opportunity ahead: training the next generation of engineers who have the judgment to direct these AI tools and the instinct to reach for them.”
The competition’s clearest finding: engineering experience is a multiplier, and the human factor remains a vital element. Mastering the first principles and fundamental concepts breeds good engineering judgment and the ability to navigate strings of tough decisions in the face of incomplete information. And when it comes to building safety-critical physical systems, nothing can replace human hands and human accountability.
“JARVIS has shown that AI copilots can have a multiplicative effect on engineering productivity, with judgment and first-principles thinking serving as the key differentiators among teams,” adds teaching assistant Kyle Woody.
But the implications of AI in aerospace are significant. If small teams using well-managed AI copilots can compress design-build-test cycles from years to weeks, the consequences for workforce structure, R&D timelines, and competitive dynamics could be substantial. The students who tackled the JARVIS Challenge are among the first engineers to grapple with those stakes not as a thought experiment, but in a machine shop, with a jet engine on the test stand.
“JARVIS highlighted the power of AI in the design of physical systems,” says Cordero, associate director of the MIT Gas Turbine Laboratory. “But it also showed that the key to unlocking that power is education, through coursework, internships, and hands-on extracurriculars like MIT Motorsports and Rocket Team. Performance in JARVIS correlated strongly with year in school. My main takeaway is that in the AI era, education is more valuable than ever.”
Generate single title from this title The best time to be a teacher: Leading with AI and technology in the age of personalized learning in 100 -150 characters. And it must return only title i dont want any extra information or introductory text with title e.g: ” Here is a single title:”
Write an article about
Key points:
- The human element remains the beating heart of education
- How districts can build a shared AI structure
- Three ways school districts can build a sustainable AI framework
- For more news on teaching and AI, visit eSN’s Digital Learning hub
In an era defined by rapid technological advancement and an unprecedented explosion of information, public education stands at a pivotal and exciting crossroads.
As Superintendent of the Van Buren School District, I have the profound privilege of leading educators who are reimagining what is possible for our students every single day. After more than a decade in this role and a career spanning teaching, principalship, and central office leadership, I am more convinced than ever: This is the best time in history to be a teacher.
For generations, dedicated educators have poured their hearts into meeting the individual needs of every child who walked through their classroom door. Yet the practical realities often forced painful compromises—limited time for deep personalization, one-size-fits-all pacing guides, and assessments that arrived too late to meaningfully adjust instruction. The information age has changed everything. Artificial intelligence and generative tools have removed many of those historical constraints, placing extraordinary power directly into the hands of classroom teachers.
Quickly identifying academic struggles
Consider the power now available to identify academic weaknesses or struggles with speed and precision that was unimaginable even five years ago. Teachers can use AI to analyze patterns in student work, flag misconceptions in real time, and surface early indicators of disengagement or learning gaps. What once required weeks of careful observation and multiple failed assessments can now be illuminated in moments. This early detection is not about replacing teacher judgment—it is about arming teachers with insights so they can intervene with compassion and precision before a struggling student loses confidence or falls further behind.
Personalized learning plans at scale
Identification is only the starting point. The real transformation lies in the ability to build personalized and customized learning plans efficiently and at scale. A teacher no longer needs to choose between meeting the needs of the advanced learner and the student who needs additional support. With well-crafted prompts and AI assistance, educators can generate differentiated pathways, scaffolded resources, interest-based extensions, and targeted interventions tailored to each child’s specific profile. Custom assessments that monitor growth in real time—formative checks, adaptive quizzes, and progress trackers aligned exactly to the skills being taught—become practical rather than aspirational.
Powerful, accessible tools for every teacher
What makes this moment truly historic is that many of the most powerful capabilities are free or readily accessible to teachers through AI and large language models.
- Claude (from Anthropic) offers sophisticated reasoning and a dedicated “Learning Mode” designed to guide students toward deeper understanding rather than simply providing answers. Teachers use it to design rich lessons, create nuanced rubrics, generate personalized feedback at scale, and build interactive learning experiences.
- Replit brings AI-assisted coding, project-based learning, and real-time scaffolding into the browser. Students build actual applications while teachers receive tools to personalize instruction and provide targeted feedback without needing to be coding experts themselves.
- Lovable empowers both educators and students to create fully functional websites and applications simply by describing what they want in natural language. This turns abstract concepts into tangible, shareable projects that develop creativity, problem-solving, and 21st-century skills in powerful ways.
These are not distant future technologies. They are available right now, often at no cost to verified educators and classrooms.
Unprecedented impact on learning outcomes
Never before in the history of education have teachers and school officials been able to leverage technology in such a direct, scalable, and impactful way to improve learning outcomes. We can finally move beyond the industrial model of education toward something far more human and effective: responsive, personalized, mastery-based learning that meets students where they are and propels them forward.
The irreplaceable human element
Yet—and this is critical—we cannot and must not rely on technology alone. The human element remains the beating heart of education. A teacher’s discernment, empathy, cultural competence, ability to build trusting relationships, and professional judgment are irreplaceable. AI can generate a thousand lesson variations, but only a skilled educator can decide which one will inspire this child on this day in this community. Technology is a powerful co-pilot; the teacher must always remain the pilot.
This is precisely why I urge every educator—especially our most experienced veteran teachers—not to fear AI and technological advancements. These tools do not diminish your expertise. They amplify it. They liberate you from hours of repetitive planning and grading so you can devote more energy to the irreplaceable work only humans can do: sparking curiosity, nurturing resilience, modeling wisdom, and forming the relationships that literally change the trajectory of a child’s life.
A veteran teacher who thoughtfully integrates these capabilities is not being replaced—they are becoming more effective than they have ever been before.
Leading forward in Van Buren Schools
In the Van Buren School District, we are committed to leading this transition with both enthusiasm and wisdom. That means providing high-quality, ongoing professional development that builds AI fluency alongside strong pedagogy. It means developing clear, practical guidelines for responsible and ethical use. It means ensuring every teacher has equitable access to tools and the confidence to experiment. Most importantly, it means keeping our north star firmly fixed on what matters most: deeper learning and brighter futures for every student we serve.
The challenges of the information age are real—navigating misinformation, managing attention, supporting mental health, preparing students for a rapidly evolving workforce. But the tools we now possess to meet those challenges are equally real and unprecedented in their power.
When we combine the timeless art of great teaching with the new science of artificial intelligence, we create classrooms where every student can truly thrive.
This is not the end of teaching as we have known it. It is the beginning of teaching as it was always meant to be—deeply personal, joyfully creative, profoundly human, and more effective than ever before.
I invite every educator, parent, and community member to lean into this moment with curiosity and courage. Start small. Experiment boldly. Ask constantly: How can this tool help me better serve the children in front of me today?
The future of education in Arkansas and across our nation is not something happening to us. It is something we are actively building—together—right now, in classrooms across Van Buren and beyond.
.Organize the content with appropriate headings and subheadings ( h2, h3, h4, h5, h6). Include conclusion section and FAQs section with Proper questions and answers at the end. do not include the title. it must return only article i dont want any extra information or introductory text with article e.g: ” Here is rewritten article:” or “Here is the rewritten content:”
Signs your commercial LED lights need maintenance
Commercial LED lights can last a long time, but they still need checks. Warehouses, offices, stores, and parking areas depend on steady light for daily use.
Small lighting issues can turn into larger repair needs when left alone. This guide covers signs that may point to needed maintenance.
Flickers or Uneven Light Levels
A light that flickers during work hours can make the space feel poorly maintained. A commercial led lighting contractor can check drivers, wiring, controls, and fixture age before parts fail.
Uneven light may also show up across aisles, work areas, or parking spaces. This may affect comfort and visibility during daily tasks.
Flicker can come from loose connections, poor dimmer matches, or power quality issues. It can also happen when older fixtures reach the end of service life.
A quick review helps find the cause before more lights act up. This may help improve light quality across the space.
Buzzing Sounds or Heat Near Fixtures
LED fixtures should run quietly during normal use. Buzzing sounds may point to driver issues, loose parts, or control problems. Heat around a fixture can also show stress in the system.
A hot fixture may have poor airflow or a failing internal part. Dust buildup can also trap heat near the unit. A reliable service provider may help inspect fixture condition, controls, warranty options, and replacement needs. Early service can reduce the chance of sudden outages.
Controls Do Not Respond Well
Lighting controls can affect how the full system works. Sensors, timers, dimmers, and switches may stop responding as expected. Lights may stay on after hours or fail to turn on in occupied rooms. This can waste power and frustrate staff.
Control issues may come from sensor placement, settings, wiring, or network faults. A commercial led lighting contractor can test the full control path.
This helps separate fixture trouble from control trouble. Proper settings may help improve both comfort and energy use.
Color Shift or Lower Light Output
Another sign of needed maintenance is a change in light color or brightness. LED technology is designed to provide steady, consistent output, but performance can change over time.
Some fixtures may begin to look yellow, blue, purple, or dull compared with others in the same area. This color shift can make a space look uneven, outdated, or poorly maintained.
Lower light output can also happen slowly. Staff may not notice the change at first because the space gets dimmer over months or years. Dust on lenses, aging LED chips, driver wear, heat damage, or poor fixture quality can all reduce output.
A lighting maintenance review can include light level readings, fixture inspection, and lens cleaning to determine whether the system needs cleaning, repair, re-aiming, or fixture replacement.
Dark Spots Across Work Areas
Dark spots can create trouble in spaces that need steady visibility. They may appear in warehouses, retail aisles, stairwells, or exterior paths. Sometimes, one failed fixture causes the problem. Other times, dirt, lens damage, or layout gaps reduce light output.
A dark area can make work slower and less comfortable. It can also make a property feel less maintained. Checking fixture height, beam spread, and lens condition can help solve the issue. Maintenance may restore better coverage without a full system change.
Areas That Need Extra Review
Some locations show light problems faster. Key areas include:
- Parking lots and garages
- Loading docks and storage zones
- Hallways and stairwells
- Customer entry points
These areas may need regular checks because they support access and safety.
Higher Energy Use or More Service Calls
A sudden rise in energy use can suggest a lighting system problem. Lights left on, faulty controls, or older fixtures can add extra cost. Modern LED technology can help lower energy use when fixtures and controls are working correctly.
More frequent service calls may also show that parts are reaching the end of their life. Tracking these patterns helps teams plan repairs better.
Maintenance records can reveal repeat issues by area or fixture type. This makes it easier to decide between repair and replacement. Smart lighting technology may also help identify usage patterns, control issues, or zones that need attention.
A planned review can help reduce emergency calls. It also keeps commercial spaces brighter, safer, and easier to manage.
LED maintenance is easier when signs are caught early. Flicker, dark spots, heat, control issues, and rising energy use all deserve attention. Modern lighting technology can perform well for years, but it still needs routine inspection to stay reliable.
Regular checks may help extend fixture life and support better daily visibility. Newer control technology can also help teams spot usage problems before they become larger repair needs. A simple service plan can keep commercial lighting more reliable across the property.
Related stories you might also like…
AI agents create virtual playgrounds to help robots get crucial training data | MIT News
Robots walking down the street, surrounded by astounded onlookers, is an increasingly common sight. But these machines aren’t yet the do-it-all assistants you’d want working in a kitchen or factory, and a major bottleneck is data. Much like humans, robots learn best by experience. The challenge is that it’s labor-intensive and time-consuming to physically teach these machines so many actions across different settings.
“One natural idea is to use simulation as a training ground. While there has been significant progress over the last few years in the physics engines that power robotics simulators, one of the remaining challenges has been creating sufficiently rich and diverse simulation content to capture the complexity of the real world,” says Russ Tedrake, the Toyota Professor of Electrical Engineering and Computer Science (EECS), Aeronautics and Astronautics, and Mechanical Engineering at MIT, and a principal investigator at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).
It turns out that AI agents, or semi-autonomous programs that “think” and complete well-defined tasks, could help produce the lifelike virtual settings that robots need. The new “SceneSmith” system developed by researchers at MIT CSAIL and Toyota Research Institute uses three agents to piece together the objects, walls, and overall look of a 3D scene. Its recreations of indoor spaces such as restaurants, bedrooms, and hotels are more realistic and detailed than prior systems, helping robots practice skills and try out different ways of doing tasks before they’re powered on. In turn, engineers save time on real-world testing.
The agents have a sense of how everyday places are supposed to look because they each call on a multi-modal system called a vision-language model (VLM), specifically the state-of-the-art VLM GPT-5.2. It’s trained on lots of text and images from the internet to handle more visual prompts. This advanced model gives each agent a sort of spatial knowledge: First, a “designer” agent generates the elements of a scene, then a “critic” advises whether it looks realistic, and finally, an “orchestrator” manages their back-and-forth, deciding when the design is done. Once the three VLMs wrap up their creative collaboration, the scene is ready to load directly into physics simulation software.
“We’ve found that the system can construct 3D scenes the way a human designer would,” says MIT EECS PhD student Nicholas Pfaff, a CSAIL researcher and a lead author on a paper with Tedrake presenting the work. “We made over 1,300 scenes using a leading VLM that has internet-scale priors, and it made insanely creative and diverse arrangements. I hadn’t taught the system to do that in the prompts; it just improvised.”
Talk to my agent
Thanks to VLM agents, you can ask SceneSmith to do things like “generate a garage with a car, a workbench, tires stacked in the corner, and a ladder against the wall,” and get a virtual playground rich with objects a robot can tinker with. These rooms are decorated with up to six times more items per scene than prior methods, making them great for helping robots learn skills such as putting a cup in the sink, placing fruit on plates, and moving a soda can from a shelf to a table.
With so many rich virtual environments handy, you can evaluate whether your robot is ready for deployment without so much trial and error in the physical world. The researchers tested out different action plans (also called “policies”) in SceneSmith’s digital worlds, generating 100 unique spaces in the process. A VLM agent evaluated each attempt, and it found the robot’s plans were faulty, with the machine often failing at its chores. Humans agreed with the model’s verdicts over 99 percent of the time, which could help roboticists weed out flawed approaches in simulation before a robot moves in the real world.
But how realistic are these virtual worlds, really? It can be difficult to prove outright, so the researchers approached the question from several angles. The most telling test: they dropped a pretrained robot policy — an AI controller trained largely on real-world data, which had never seen a SceneSmith scene — into the generated environments. In one test, users told the system to “take the apple from the bowl and place it onto the cutting board,” and the simulated robot did exactly that. If the scenes didn’t closely resemble the real settings the policy had learned from, it simply wouldn’t have worked.
The team also teleoperated robots through the virtual spaces, guiding them to open cabinets, put away bottles, and navigate between rooms. Their experiments revealed that the environments hold up under sustained physical interaction, expanding beyond visual inspection.
Behind the scenes
The agents that SceneSmith uses each have a well-defined role in the generative process, fleshing out scenes in stages. They essentially create a floor plan and bring it to life.
Let’s say you wanted to create a scene similar to the first floor of a house. The “designer” VLM would start with a general layout, which the “critic” reviews, and then the “orchestrator” signs off. The agents repeat this approach for each step: adding furniture, placing objects on walls and then ceilings, and finally, dropping in objects that robots can manipulate. For example, the VLMs can add cabinets that the robots can open and close — an articulated item, which prior baselines didn’t often have.
At each stage, the second VLM ensures the scene is practical, advising that a bathtub is removed from a living room, for example. The third VLM ensures a high-quality scene is generated, even taking the design process a few turns back if the visuals aren’t up to par. Once the three VLMs wrap up their creative collaboration, the mechanics of the physical world are added via simulation software.
With a sound understanding of how rooms should look, where objects should be placed, and real-world physics, SceneSmith has a noticeable edge over prior methods. Compared to scene-generation baselines such as “HSM” and “Holodeck,” SceneSmith made environments with more objects, including a private office, a pottery store, and even a Minecraft-themed gaming room.
SceneSmith was also a favorite among over 200 users. They found the system’s visuals to be more realistic over 90 percent of the time. They also observed that, generally speaking, it followed prompts more closely than other approaches did. In other words, it was the best at generating the virtual playgrounds users actually wanted to see.
A system of many talents
Realism, diversity, and richness are all strong suits for SceneSmith, even when it comes to generating individual 3D objects. You can prompt it to create a rolling serving cart, and it’ll make a 2D image that it then turns into a detailed model with physical properties like mass, friction, and inertia.
Such a detailed process does come with a speed trade-off, though. It can take multiple hours to produce a single scene because the agents are creating and closely scrutinizing each object. With more computing power, the system could see dramatic increases in efficiency. CSAIL engineers are also hoping to expand to deformable objects (like sponges), should extensive 3D libraries become available.
“SceneSmith represents a significant advance in this regard by providing an agentic framework for generating simulation-ready indoor environments just from a simple text prompt,” says Jeremy Binagia, an applied scientist at Amazon Robotics who wasn’t involved in the research. “It advances the state of the art in several ways, including pushing the limits of the density of objects in the simulated environment, ensuring that all of the objects are physically accurate (versus just being visually realistic), and creating assets that are not constrained to a fixed library, since they can be generated via text-to-3D.”
Pfaff and Tedrake wrote the paper with Thomas Cohn SM ’24, an MIT PhD student and CSAIL researcher; and Toyota Research Institute roboticists Sergey Zakharov and Rick Cory SM ’08, PhD ’10. Their work was supported, in part, by Amazon, the U.S. Office of Naval Research, the Toyota Research Institute, and the U.S. National Science Foundation.
The team presented their findings as a spotlight at last week’s International Conference on Machine Learning.
