In keeping with the Linked Solutioning theme of our Partnership Model, we plan to enhance our solutions, products and services by embedding electronic componentry provided by the key players in IoT and smart PCB technologies. The Smart Monitoring protocol consisting of the CONNECT-DETECT-PROTECT theme enables personnel safety, situational awareness, environmental assessment, operational management and functional efficiency without being authoritative, overbearing and intruding on privacy.

Smart monitoring is the judicious use of the different sensors to collect data using edge devices in a strategic and efficient manner. The sensors used for smart monitoring can vary depending on the application, but some common types include vision sensors, temperature sensors, humidity sensors, pressure sensors, proximity sensors, and motion sensors.

The data collected by these sensors can be analyzed in real-time or stored for later analysis using big data analytics and machine learning algorithms. By analyzing the data, patterns and trends can be identified, allowing for proactive measures to be taken to prevent issues, issue warnings or optimize processes. Thus, smart monitoring is the front end of our holistic data engineering theme at Numorpho Cybernetic Systems (NUMO) to enable actionable intelligence by converting meaning to motion.

Smart monitoring can be used in various industries and applications, such as:

• enabling safety in industrial, construction, first responder and mining settings,
• monitoring equipment in manufacturing plants,
• tracking inventory in retail environments,
• monitoring traffic patterns in smart cities,
• ensuring safety in local commutes, and
• intelligently monitoring patients in healthcare settings.

The thesis behind the data engineering for smart monitoring is to stage the data in different “intervals” – device, edge, fog and cloud/data center for ease of processing, proximity for reuse and for enabling security and privacy.

In relation to human perception, smart monitoring can provide insights into the environment and conditions around us that are not immediately perceptible to our senses. For example:

• Vision sensors can recognize, enable quality inspection and prevent accidents.
• Air quality sensors can detect pollutants in the air that are not visible or noticeable by smell, providing information on potential health hazards.
• Motion detectors can indicate collisions, falls and other impactful situations
• Similarly, temperature and humidity sensors can provide insights into the comfort levels of a room or building that may not be immediately perceptible to individuals.

In this whitepaper we will summarize our thesis for smart monitoring and our partnerships with:

• Ardunio for their Nicla and PRO line sensors
• MxD for their sensor kit
• Sony AITRIOS for their edge-based vision sensing (TBD)
• Omron for their industrial and environment-based sensors
• Avnet for their subject matter expertise with sensors

TABLE OF CONTENTS

• Functional Setups
  • Manufacturing
  • Infrastructure
• Connect-Detect-Protect
  • Interaction Diagram for Data Engineering
  • Smart Helmet Considerations
• Partnership Projects
  • A. Project Tiramisu with Arduino PRO
  • B. Project Elevate with MxD’s Sensor Kit
  • C: Potential Project with Sony AITRIOS
  • D. Project SAFE @MxD: Boeing + Northwestern
• Use Cases
  • UC01 – Motion and Environment Sensing using Arduino Nicla Sensors
  • UC02 – Brownfielding: Retrofitting sensors to existing industrial equipment and monitoring
  • UC03 – Measurement of overall equipment effectiveness using Sony AITRIOS IMX 500 AI Vision sensor (TBD)
• Project Phases
  • Phase 1
  • Phase 2
  • Phase 3

FUNCTIONAL SETUPS

I. Manufacturing

Personas/Actors

CONNECT-DETECT-PROTECT

Illustrated below is how we collaborate with Partners to embed sensors and other electronic boards for our smart helmets, e-mobility solutions, smart city services and process automation:

Smart Monitoring, as indicated in the diagram above, follows the theme of Connect-Detect-Protect, which will be our methodology for employing a structured approach to our progression in electronic componentry and its future data engineering. It will also enable detailed blue printing of our use cases.

Connect refers to the act of integrating electronic components, sensors, and devices into solutions, products, and services. This involves installing sensors to our products, connecting devices to the cloud, integrating wireless protocols, and using other methods to enable communication and data exchange between different systems. By connecting devices and systems, we create a more integrated and efficient ecosystem that enables new capabilities and functionalities making the products smart and connected.

Detect refers to the act of using sensors and other electronic components to detect data and information about the environment, systems, and users. This involves using sensors to detect data on vision, temperature, humidity, pressure, location, motion, and other environmental factors. By detecting, we gain insights into:

• • how the systems are operating,
  • how users are interacting with them, and
  • what factors are affecting their performance.

Protect refers to the act of using structure and materials, electronic components, sensors, and other technologies to protect users and assets from harm. This involves:

• • engineering the helmet for impact
  • using sensors to detect data on things like air quality, noise levels, and other environmental factors that could affect user health and safety.

It also involves introspecting the proximity around to ensure that machines and systems are operating safely and efficiently, reducing the risk of accidents or breakdowns.

The combination of Connect, Detect, and Protect helps create more integrated, efficient, and safe systems that improve the lives of customers and users. By integrating electronic components and sensors into our solutions, products, and services, we can create a more connected and responsive ecosystem that enables new capabilities and functionalities, thus conforming to our theme of “Everything Connected.”

Smart Helmet Considerations

Embedding electronic boards into the folding helmet is an exciting use case. Here are some potential considerations for selecting and integrating electronic componentry into the helmet:

1. Size and weight: Since the helmet needs to be wearable, it’s important to select electronic components that are small and lightweight. This will ensure that the helmet remains comfortable and doesn’t interfere with the user’s movement or vision.
2. Sensors: To enable vision, biometric, and environment sensing, we need to select the appropriate sensors for each use case. For example, a camera or infrared sensor for vision, heart rate or blood pressure sensor for biometric sensing, and temperature or humidity sensor for environment sensing.
3. Communication: The sensors will need to communicate with each other and with the embedded processor in the helmet. We need to select communication protocols that are reliable, low-power, and suitable for the use case. For example, Bluetooth Low Energy (BLE) or Zigbee could be good options.
4. Power: The helmet will need a power source to run the sensors and embedded processor. We need a power source that is small, lightweight, and can provide enough power for the required use cases. For example, a small lithium-ion battery could be suitable.
5. Embedded processor: The embedded processor will need to process the data from the sensors and enable voice-based actionability. We need to select a processor that is powerful enough to handle the required tasks, but also low power to conserve battery life. For example, an ARM Cortex-M processor could be a good option.
6. Voice-based actionability: If you plan to enable voice-based actionability in the future, you’ll need to integrate a microphone and speaker into the helmet. You’ll also need to select a voice recognition technology that is accurate and can work in noisy environments.

Overall, embedding electronic boards into the folding helmet for the Hercules variant is a challenging but exciting use case. By carefully selecting and integrating electronic componentry, we can create a helmet that is not only safe and comfortable but also provides advanced sensing and voice-based functionality.

PARTNERSHIP PROJECTS

A. PROJECT TIRAMISU with Arduino PRO

Our partnership with Arduino consists of embedding their PRO line of sensors starting with the Nicla family to embed them in our smart helmets to enable situational monitoring in industrial, commercial and recreational use. Here is a whitepaper published by Arduino that details our interactions:

• Case Studies – Numorpho (arduino.cc)
• WebBLE Dashboard

Project Tiramisu consists of a series of use cases to enable our smart monitoring functionalities by embedding electronic componentry (in this case Arduino Nicla sensors) and activating the different sensor constructors into our themed solutions in industrial safety and automation.

Our use cases of embedding electronic boards would be for our folding helmet and the connected ecosystem of the factory floor. Our plan is to include geocentric, biometric and environment sensors initially with vision and voice based actionability in the future.

Project Tiramisu will be our first foray into embedding our products with sensors to make them smart and connected. In a series of Use Cases with MVP generations, we will add capabilities into our folding helmets to make them connect, detect and protect for different use cases.

Feature list for Activation

To enable safety, protection, and operational use of the helmet, we will program the following features:

1. Accelerometer: This sensor can detect the orientation and movement of the helmet, which can be used to detect impacts or falls. This can trigger alerts or notifications to the user or a monitoring system.
2. Gyroscope: A gyroscope can measure the helmet’s angular velocity, which can provide data on the direction and speed of movement. This can be used to detect sudden changes in direction or rotation, which can indicate an impact or collision.
3. Magnetometer: A magnetometer can detect changes in the magnetic field, which can be used for navigation and orientation purposes. This can help the user to determine their location or direction.
4. Temperature and Humidity Sensors: These sensors can provide data on the environmental conditions inside the helmet. This can help to ensure that the user is comfortable and safe, especially in extreme weather conditions.
5. Visualization: Visualize the analyzed data to gain insights and make informed decisions. This can involve creating charts, graphs, and other visualizations to represent the data in a meaningful way.
6. Predictive Maintenance: Use the analyzed data to predict when maintenance is required for the helmet or its components. This can help prevent equipment failure and ensure the safety of the user.
7. Continuous Monitoring: Continuously monitor the sensor data and adjust your analysis techniques as required. This can involve implementing real-time data processing and analytics to ensure that the system is always current with the latest data.

B. PROJECT ELEVATE WITH MxD’s Sensor Kit

Here is a pertinent post from our partner organization MxD, that affirms the need to incorporate smart technologies in Small and Medium Manufacturing (SMM) companies.

• https://www.linkedin.com/feed/update/urn:li:activity:7164352205150433280/
• Smart Manufacturing 101

It showcases how MxD’s sensor kit was employed at 10 small factories providing “simple no-code solutions” for pertinent use cases that enabled process optimization and cost savings and “experience the benefits of digitization.”

It also refers to a larger article by SME, where we have been members of for several years that sets the stage for smart manufacturing and how to get started. The list includes using digital twins, AI, cyber-security and augmented reality solutions all of which we at Numorpho Cybernetic Systems (NUMO) are actively pursuing as part of our process engineering platform Mantra M5 to enable make, manage, move, market and maintain.

Digital and smart manufacturing is a need as we begin to rely more on local and on-shore production capabilities. It is even more pertinent due to fact that the workforce is aging – the silver tsunami effect – and the need to automate is ever more pertinent. With the advent of new engineering like Additive Manufacturing we are able to build complicated parts that require a different skill set to design and make.

At NUMO, we have a scaled maturity model for incorporating Industry 4.0. It follows brownfield retrofitting, greenfield lift and shift and/or bluesky disruptions that enable blueprinting of initiatives using our unique Digital Twine Reference Architecture to theme out process Digital Twins that are intelligent and evolve with time to match with the progression of new needs.

Our forays into generative AI will help smart process engineering by enabling actionable intelligence to convert meaning to motion using the rigor of analytics and simulations.

#MXDSensorKit

Digital and smart manufacturing is a need as we begin to rely more on local and onshore production capabilities. It is even more pertinent due to fact that the workforce is aging – the silver tsunami effect – and the need to automate is ever more pertinent. With the advent of new engineering like Additive Manufacturing we are able to build complicated parts that require a different skill set to design and make.

MxD has a unique sensor kit set that enables brownfielding of manufacturing plants – retrofitting equipment with sensors to make them smart for IoT and other Industry 4.0 protocols to better communicate.

From MxD’s Lender Agreement:

The Borrower shall use the Equipment solely in the conduct of its business, and for the sole purpose of utilizing the Equipment to improve its digital manufacturing processes and other various scenarios contemplated thereof, in protected and unprotected environments, and in compliance with all laws, rules, and regulations.

By agreeing to the loan of equipment, Borrower agrees to participate in MxD’s Sensor Kit Program (“Program”), via an Alliance Partner program sponsor. Participation in the program requires that the Borrower:

i. Agree to use the Kit,
ii. Inform MxD how they use the Kit,
iii. Provide detailed feedback on the functionality of the Kit,
iv. Provide detailed information on the Kit’s performance toward solving an existing problem for the Borrower, and
v. Provide detailed feedback on the degree to which the Kit added value to the Borrower’s operations.
vi. Provide the above information in the form of a report by EOY of the year in which the sensor kit is received.

This fits well with Numorpho’s CONNECT-DETECT-PROTECT theme. A comprehensive smart monitoring application could be devices by combining the data from static (in place sensors) with dynamic/roving sensors like those attached to our helmets and other ARVs.

• 16-02-07 Final Report Project NEW DIGITAL TRICKS FOR TRUSTED FRIENDS

In this project, we utilize MxD’s sensor kit to elevate the SMM facilities to retrofit sensors to existing equipment providing “simple no-code solutions” for pertinent use cases that enabled process optimization and cost savings and “experience the benefits of digitization.”

USE CASES

Use Cases are scenarios of system behavior and express a business need. They are defined broadly in this context and consume information and data independently from each other. Our use cases tease out smart monitoring functionalities by utilizing our partner ecosystem (Linked Solutioning) to embed sensors and services into our smart and connected products and solutions.

Based on the needs of the use case, we will have a configurable setup to implement the use case by including the sketch code. The sensors fitted to our mobile solutions will be encased in housings to protect it from the elements and enable charging and firmware updates without the need to open the casing.

UC01 – Motion and Environment Sensing using Arduino Nicla Sense ME

OBJECTIVE: Monitor movement and environmental parameters utilizing sensors embedded in mobile products that are connected wirelessly to local nodes to facilitate industrial safety, well care at home and nursing facilities, first responder awareness, and recreational activities. This will enable proactive safety management using sensor data, its coordinated data engineering and provide for alerts and appropriate response.

PRECONDITIONS/NEEDS:

• Movement monitoring,
• Geo coordination,
• Fall detection,
• Impact sensing,
• Environmental monitoring.

Core Functions:

• Connect product with sensor devices – embedded and on the edge.
• Detect and transfer data from device to local nodes.
• Protect occupant from harm.

Post-Conditions:

• Transfer appropriate data to cloud.
• Determine analytical model for predictive outcomes.
• Provide for actionable intelligence.

Data Flow:

INDUSTRIAL WORKER SAFETY USE CASE

https://docs.arduino.cc/tutorials/nicla-sense-me/getting-started

https://www.electromaker.io/blog/article/product-of-the-week-arduino-nicla-sense-me

This is a small sensor board that includes an accelerometer, gyroscope, magnetometer, and environmental sensors such as temperature and humidity. By integrating the Nicla Sense ME board with the Nicla Vision board, our aim is to create a more advanced safety system for the helmet, with features such as geo coordination, fall detection, impact sensing, and environmental monitoring.

Sensors on the Nicla Sense ME
The Nicla Sense ME’s BHI260AP sensor includes a 6-axis IMU, that we will use to give us accelerometer and gyroscope readings. An accelerometer is an electromechanical device used to measure acceleration forces. Such forces may be static, like the continuous force of gravity or, as is the case with many mobile devices, dynamic to sense movements or vibrations. On the other hand, a gyroscope sensor can measure and maintain the orientation and angular velocity of an object. Gyroscopes are more advanced than accelerometers, as they can measure the tilt and lateral orientation of an object, whereas an accelerometer can only measure its linear motion.

Sensor Types
To access the Arduino Nicla Sense ME sensor’s data, you will need to declare the type of sensor you want to use (they are called constructors), since they have different data functions that return data (e.g Accelerometer will have X, Y and Z values).

Engineering Workplan

Intentions:

• • Vented Housing to encase the Nicla Sense ME, battery and power switch
  • Enable sensor turn on/off. Future updates will have a sleep/wake up mode to conserve battery.
  • 3.7V Coin cell battery with enclosure and leads
  • Enable battery charging
  • Enable firmware updates (future)
  • Build sketch code (temperature, CO2, motion, collision)
  • Create user interface (BLE, Mobile)

Sensors are housed in encasements that fit the Nicla Sense ME along with the 3.7V coin cell battery and its enclosure, the battery charging board and a toggle switch to enable turning on the sensor or enabling charging (off position).

Shown below is the circuit diagram to enable the Nicla Sense ME to be charged whilst in the encasement.

The purpose of this circuit is to enable switching between battery charging by the TP4056 charging board and battery discharging to power the Nicla Sense Me Chip. The two-position slide switch enables this switching capability by closing the desired circuit. In the above diagram, if the switch is toggled to the left it will close the charging circuit, connecting the TP4056 charging board to the 3.7V coin cell. Power is provided through the USB C/USB micro on the charging board. Alternatively, if the switch is toggled to the right, this closes the discharging circuit, causing the Arduino Sense Me chip to be powered by the 3.7 V coin cell.

UC02 – Brownfielding: Retrofitting sensors to existing industrial equipment and monitoring

MxD’s Sensor Kit Package: MxD (Manufacturing times Digital) offers a unique sensor kit set designed for brownfield retrofitting of manufacturing plants. This kit allows for:

1. Easy installation of sensors on existing equipment
2. Conversion of legacy machinery into smart, IoT-enabled devices
3. Integration with Industry 4.0 protocols
4. Real-time data collection from manufacturing processes
5. Improved communication between different parts of the manufacturing system

Key features likely include:

• Plug-and-play sensors for various parameters (temperature, vibration, pressure, etc.)
• Edge computing capabilities for local data processing
• Secure connectivity options (Wi-Fi, cellular, or wired)
• Compatibility with common industrial communication protocols

Numorpho’s CONNECT-DETECT-PROTECT: This theme encapsulates Numorpho’s approach to creating smart, interconnected safety solutions. In the context of our innovative helmets and other ARVs (Autonomous Robotic Vehicles), this involves:

1. CONNECT: Establishing a network of sensors and devices that communicate seamlessly
2. DETECT: Using advanced sensors to monitor environmental conditions, user status, and potential hazards
3. PROTECT: Leveraging collected data to enhance safety measures and respond to threats in real-time

Comprehensive Cybernetic Fabric for Data Engineering and Smart Monitoring:

By combining MxD’s static sensor infrastructure with Numorpho’s dynamic, embedded/wearable sensors, we can create a comprehensive cybernetic fabric that offers unparalleled monitoring and data analysis capabilities:

1. Multi-layer Sensing Network:
   • Static Layer: MxD’s sensors on manufacturing equipment provide constant monitoring of machine status, environmental conditions, and production metrics.
   • Dynamic Layer: Numorpho’s smart helmets and ARVs offer mobile sensing capabilities, providing data from various locations within the facility.
2. Comprehensive Data Collection:
   • Equipment Performance: Real-time monitoring of machinery efficiency, wear, and potential failures.
   • Environmental Monitoring: Tracking of temperature, humidity, air quality, and other relevant factors across the entire facility.
   • Worker Safety: Continuous monitoring of worker locations, vital signs, and exposure to potential hazards.
   • Process Optimization: Detailed data on production processes, material flow, and quality control.
3. Advanced Data Integration:
   • Fusion of static and dynamic data sources to create a holistic view of the entire manufacturing ecosystem.
   • Real-time data aggregation and analysis to identify patterns, anomalies, and optimization opportunities.
4. Predictive Analytics and AI:
   • Utilization of machine learning algorithms to predict equipment failures, safety risks, and production bottlenecks.
   • AI-driven decision support for optimizing processes, scheduling maintenance, and enhancing safety measures.
5. Responsive Safety Systems:
   • Immediate alert systems for potential hazards detected by either static or mobile sensors.
   • Automated safety protocols triggered by data from the integrated sensor network.
6. Customized Dashboards and Reporting:
   • Real-time visualization of key metrics and alerts for different stakeholders (operators, managers, safety officers).
   • Customizable reports for compliance, performance analysis, and continuous improvement initiatives.
7. Scalable and Flexible Architecture:
   • Easy integration of new sensors and devices as the facility expands or requirements change.
   • Adaptable data processing capabilities to handle increasing data volumes and complexity.
8. Cybersecurity Measures:
   • Robust security protocols to protect sensitive manufacturing and personal data.
   • Secure communication channels between all elements of the cybernetic fabric.
9. Continuous Learning and Improvement:
   • Feedback loops that allow the system to learn from historical data and improve its predictive capabilities over time.
   • Regular updates to algorithms and models based on new data and changing manufacturing conditions.

This comprehensive cybernetic fabric creates a synergy between MxD’s brownfield retrofitting capabilities and Numorpho’s innovative mobile sensing solutions. It offers a holistic approach to smart manufacturing and worker safety, providing unprecedented levels of insight, control, and predictive capabilities. This integrated system not only enhances operational efficiency and safety but also positions manufacturing facilities at the forefront of Industry 4.0 innovation.

FIRST USE

Thank you MxD for your Sensor Kit program and thank you VoxSomnia for installing it at our premises at mHUB .

Albeit the sensor kit was intended to elevate traditional manufacturing to brownfield them to Industry 4.0, our greenfield use case at Numorpho Cybernetic Systems (NUMO) is a perfect example where the sensor kit enhances the monitoring capabilities of new machines. In our first use case we have attached it to our Bambu Lab 3D printer to additionally monitor internal chamber temperature and vibrations. After our initial test run, we have continuously used it for a 15-hour print job for an amazing project that we are starting on. But that is another story…..

We plan for the sensor kit to be an integral part of our CONNECT-DETECT-PROTECT smart monitoring protocol where coordination of multi-modal data is key for safety and efficient operations in factory floors, and other infrastructure, construction and facilities management. Here we are utilizing sensors from Arduino (the Nicla family and their other PRO products), Stroma Vision and others for a concerted and engineered theme on managing operations data and enabling smart manufacturing.

Thank you:

Jacob Smith, Brian Townley, @Erik Rancatore from MxD.

Andy MacDonald, Rebecca Tiwet, Eric Lovelady from VoxSomnia

#MXDSensorKit

UC03 – Measurement of overall equipment effectiveness

• Example scenario: FactoriesMeasure the overall equipment effectiveness (OEE) without customizing existing equipment. Cameras are installed for production equipment input and output locations, and edge AI analyzes the operating rate and performance.Visualizing the production efficiency of conventional equipment allows review of the production line and quantitative evaluation of the improvement effect to enhance productivity.
• https://developer.aitrios.sony-semicon.com/en/edge-ai-sensing/system-overview/
• https://developer.aitrios.sony-semicon.com/en/edge-ai-sensing/devices/
• https://developer.aitrios.sony-semicon.com/en/edge-ai-sensing/documents/get-started/

PROJECT PHASES

A GIT Repository was created to manage all the code components for the sketch code to update the sensor firmware and collaborate with the developers, test the development and roll out the code to the appropriate customers.

• Phase 1 involved
• Phase 2 involved
• Phase 3 involved

SUMMARY

From ChatGPT

Collaborating with key players in IoT and smart PCB technologies like Arduino and Bosch Sensortec can help integrate sensors and other electronic components into your solutions, products, and services. This can improve functionality, increase efficiency, and enhance safety.

The Connect-Detect-Protect theme is a great way to think about how these electronic components can be used to create solutions that connect devices, detect and respond to data, and protect users and assets. For example, by integrating sensors into your e-mobility solutions, you can connect your vehicles to the cloud and detect data on things like location, speed, and battery life. This information can then be used to protect the users of the vehicles by ensuring they have enough power to get to their destination safely.

Similarly, by integrating sensors into your smart city services, you can connect devices like traffic lights, parking meters, and waste management systems to the cloud and detect data on things like traffic flow, parking availability, and garbage levels. This information can then be used to protect the citizens of the city by improving traffic flow, reducing congestion, and keeping the streets clean.

Finally, by integrating electronic components into your process automation systems, you can connect machines, detect data on things like temperature and pressure, and protect workers by ensuring the machines are operating safely and efficiently. This can improve productivity, reduce downtime, and keep workers safe.

Overall, collaborating with companies like Arduino and Bosch Sensortec to embed electronic componentry in your solutions, products, and services can help you create connected, efficient, and safe systems that improve the lives of your customers and users.

NITIN UCHIL Founder, CEO & Technical Evangelist
nitin.uchil@numorpho.com

REFERENCES

• IMX500 | Developer World (sony.com)

MxD Final Presentation: Project 22-06-01 Proactive Worker Safety for Industry 4.0 Using AI

Project Participants: MxD, Arvist, University of Buffalo, and PPC Flexible Packaging

Date: Thursday, March 13th, 2025 at 10:00 a.m. – 11:00 a.m. CT

Join MxD, Arvist, University of Buffalo, and PPC Flexible Packaging for the final presentation on MxD project 22-06-01. This presentation will demonstrate how the project team combined the latest advances in Artificial Intelligence (AI) with ergonomic risk assessment tools and specialized safety knowledge to create a safety solution which is fit for Industry 4.0 workers. The project team developed a platform using AI video analytics technology to give accurate risk assessment of ergonomics by providing real-time performance data to both manufacturers and workers.