Meca500 - SICK PLOC 2D Vision System Integration (using Python)

Introduction:

PLOC2D from SICK is an easy to use flexible part localization sensor system that can be used for Vision Guided Pick and Place Applications. PLOC 2D supports multiple interfaces including TCP/IP, EtherNet/IP and Profinet out of the box. They provide an intuitive browser based interface that allows an user to set up a job and run in matter of few minutes.

In this MecaNetwork article, we will discuss in detail about the integration of a SICK PLOC 2D Vision System with a Meca500-R3 using Python.


System Requirements:

Meca500-R3 system

SICK PLOC 2D Vision System with Power Supply

M12-D coded - RJ45 cable 

Windows or Linux PC with Python 3.6 or above and numpy installed

PLOC 2D Robot Guidance System Setup:

A complete setup guide for PLOC 2D is beyond the scope of the this article, however SICK offers a comprehensive integration guide that describes the workflow for setting up a PLOC 2D which has been attached in this article for reference.

The key steps involved are 

1. Installation

2. Calibration

In this step, the system will acquire a set of calibration images in order to estimate the lens distortion parameters. 

3. Alignment:

In this step, the system will establish the pose of the camera in relation to the robot coordinate system. Once the origin of the reference frame has been detected by the vision system, use 3 point method to calculate the vision reference frame with respect to robot's Base Reference Frame.

4. Configure Job

This step defines the shape that the system will look for

5. Run Job



SICK PLOC-2D VisionController Python Class:

This Python class is used to set up a TCP/IP communication interface with PLOC 2D vision system to send commands and receive responses.


import socket

class VisionController:
    """ Class for the SICK PLOC 2D Vision for Robot Guidance and
    use with Mecademic Robot

    Attributes
    ----------

    address : string
        The IP address associated to the SICK PLOC 2D vision system.
    socket : socket object
        Socket connecting to physical SICK PLOC 2D vision system.
    error : boolean
        Error Status of the SICK PLOC 2D vision system job.
    x  : float
        The Cartseian value in mm  for X.
    y  : float
        The Cartseian value in mm  for Y.
    z  : float
        The Cartseian value in mm  for z (incl. of offset).
    rz : float
        The Cartseian value in mm  for Rz.  (Uses Staubli's Notation)
    index  : int
        The index of the current identified part among a set of matches.
    matches  : int
        Total no of identified parts for the selected job.
    error_ID  : int
        The error ID returned by SICK PLOC 2D for the selected job.
        
    """

    def __init__(self, address):
        """ Constructor for an instance of the class SICK_PLOC2D vision system

        Parameters
        ----------
        address : string
            The IP address associated to the SICK PLOC 2D vision system
        """
        self.address =  address
        self.socket = None
        self.error = False
        self.errorID = None
        self.jobID = None
        self.x = None
        self.y = None
        self.z = None
        self.rz = None
        self.index = None
        self.matches = None
        

    def connect(self):
        """ Connects SICK PLOC 2D vision system object communication to the physical SICK PLOC 2D vision system

        Returns
        -------
        status  : boolean
            Return whether the connection is established
        """

        try:
            self.socket = socket.socket()
            self.socket.settimeout(0.1)
            try:
                self.socket.connect((self.address,14158))
            except socket.timeout:
                raise TimeoutError
            
            # Receive confirmation of connection
            if self.socket is None:
                raise RuntimeError

            return True

        except TimeoutError:
            return False
        except RuntimeError:
            return False
        
    def disconnect(self):
        """ Disconnects SICK PLOC 2D vision system object communication to the physical SICK PLOC 2D vision system

        """
        if (self.socket is not None):
            self.socket.close()
            self.socket = None

    def _send(self,cmd):
        """Sends a command to the physical SICK PLOC 2D vision system.
        Parameters
        ----------
        cmd : string
            Command to be sent.
        Returns
        -------
        status : boolean
            Returns whether the message is sent.
        """
        if self.socket is None:               #check that the connection is established 
            return False                      #if issues detected, no point in trying to send a cmd that won't reach the vision system
        cmd = cmd
        status = 0
        while status == 0:
            try:                                                #while the message hasn't been sent
                status = self.socket.send(cmd.encode('ascii'))  #send command in ascii encoding
            except:
                break
            if status != 0:
                return True                                 #return true when the message has been sent 
                                                            #Message failed to be sent, return false
        return False

    
    def _get_data(self,delay = 1):
        """Receives message from the SICK PLOC 2D vision system and 
        saves the values in appropriate variables.
        Parameters
        ----------
        delay: int or float 
            Time to set for timeout of the socket.
        """

        if self.socket is None:
            return
        self.socket.settimeout(delay)
        try:
            raw_response = self.socket.recv(1024).decode('ascii').split(",")         #read message from vision system
            if raw_response[0].find('Ok') != -1:
                self.error = False
                self.errorID = 0
                self.jobID = int(raw_response[1])
                self.index = int(raw_response[2])
                self.matches = int(raw_response[3])
                self.x = float(raw_response[4])
                self.y = float(raw_response[5])
                self.z = float(raw_response[6])
                self.rz = float(raw_response[9])
                self.score = int(raw_response[-4])
            else:
                self.error = True
                self.jobID = int(raw_response[1])
                self.errorID = int(raw_response[-1])
        except TimeoutError:
            pass
        
    def run_job(self,jobId):
        """Sends and receives with the SICK PLOC 2D vision system.
        Parameters
        ----------
        jobId : int
            Job to send to the SICK PLOC 2D vision system.
        Returns
        -------
        response : boolean
            Returns whether the job was completed without any error
        """

        cmd = "Run.Locate"+","+str(jobId)
        status = self._send(cmd)
        if status:
            response = self._get_data()
            if (not self.error):
                return True
            else:
                return False

    def locate_by_index(self,jobId,index):
        """Sends and receives with the SICK PLOC 2D vision system.
        Parameters
        ----------
        jobId : int
            Job to send to the SICK PLOC 2D vision system.
        index : int
            Index of the identified parts to be picked
        Returns
        -------
        response : list
            Returns the list of current index and x,y,z and rz values in mm and deg
        """
        cmd = "Run.Locate"+","+str(jobId)+","+str(index)
        status = self._send(cmd)
        if status:
            response = self._get_data()
            if (not self.error):
                return tuple((self.index,self.score,self.x,self.y,self.z,self.rz))

    def is_in_error(self):
        return self.error
            
            



MecademicRobot Python Class:

Mecademic Robots can be controlled via our Python API available at our Github page. The Python API has classes for both robot control on port 10000 and feedback on port 10001. 

Vision Guided Pick Python Class:

The VisionGuidedPick Python class is higher level abstraction class that handles the inetgration of the RobotControl and VisionControl as one and provides methods to perform vision guided pick and place applications. It provides a simple interface for both vision and robot control. It provides various methods that includes getting the total no of identified parts for a given vision job, pick by index, and place etc.,


def get_count(self,jobId):
        """ Retrieves the total no of identified parts from the vision job

        Returns
        -------

        counts : int
            No of identified parts
        
        """
        self.vision.run_job(jobId)
        if not self.vision.error:
            return self.vision.matches


def pick_index(self,jobId,index):
        """ Starts the pick cycle for a given index of the matched parts

        Parameters
        ----------
        jobId : int
            Vision job Id that is used to identify a part
        index : int
            Index "n" of the list of identified parts in a vision job
        """
        self.jobId = jobId
        self.index = index

        self.count = self.get_count(self.jobId)

        if self.count != 0:
            [self.current_index, self.current_index_score, self.pose_x, self.pose_y, self.pose_z, self.pose_rz] = self.vision.locate_by_index(self.jobId,self.index)
            if self.current_index == self.index:
              
                self.robot.MovePose(self.pose_x, self.pose_y, self.pose_z + self.z_offset, self.pose_rx, self.pose_ry, self.pose_rz)
                self.robot.SetCartLinVel(self.linear_speed/3)
                self.robot.MoveLin(self.pose_x, self.pose_y, self.pose_z, self.pose_rx, self.pose_ry, self.pose_rz)
                self.gripper_close()
                self.delay(0.25)
                self.robot.SetCartLinVel(self.linear_speed)
                self.robot.MoveLin(self.pose_x, self.pose_y, self.pose_z + self.z_offset, self.pose_rx, self.pose_ry, self.pose_rz)


def place(self,x,y,z,rx,ry,rz):
        """ Starts the place sequence with Approach and Retract defined with a user defined Z offset
        and user defined place pose

        Parameters
        ----------
        x : float
            X axis value in mm with respect to robot's BRF
        y : float
            Y axis value in mm with respect to robot's BRF
        z : float
            Z axis value in mm with respect to robot's BRF
        rx : float
            Rotation about X axis in degrees
        ry : float
            Rotation about Y axis in degrees
        rz : float
            Rotation about Z axis in degrees
        """
        self.robot.MovePose(x, y, z + self.z_offset, rx, ry, rz)
        self.robot.MoveLin(x, y, z, rx, ry, rz)
        self.robot.gripper_open()
        self.robot.delay(0.25)
        self.robot.MoveLin(x, y, z + self.z_offset, rx, ry, rz)


Calibrating the Vision Reference Frame with respect to Meca500's BRF:

An important step in implementing a vision guided pick and place application is calibration of the Vision Reference Frame with respect to the robot's Base Reference Frame. At Mecademic Robotics, we prefer calculating the WRF using a 3 point method. The VisionGuidedPick class offers a method to calculate the Vision Reference Frame and set it as well. Alternatively, users can also use the standalone executable to calculate the Vision Reference Frame. Reach out to support@mecademic.com for the WRF calculator exe file. 



Application Code:

Once we have all the required classes defined, then we can create an vision guided pick and place application by invoking an instance of the VisionGuidedPick class with the IP address of the Meca500 and PLOC 2D and use the methods of this class to get the total count of identified parts, pick by index and place the part.


from VisionGuidedPick import VisionGuidedPick

# Create an instance of the vision guided pick class
app = VisionGuidedPick('192.168.0.100','192.168.0.1')
# Initialize the vision and robot
app.init_robot()
app.init_vision()

# Set the config parameters
app.set_vision_ref(100,50,25,0,0,0)
app.set_offset(25)
app.set_joint_speed(50)
app.set_linear_speed(150)
app.set_place_pose()

# Get the total counts of the matched parts for the selected vision job ID
count = app.get_count(1)

# Pick the first matched part
if count is not None:
    app.pick_index(1,1)             
    app.place(-120,100,0,180,0,180)


# Get the total counts of the matched parts for the selected vision job ID
count = app.get_count(1)

# Loop through all the identified parts
if count is not None:
   for i in range(1,count+1):       
       app.pick_index(1,i)
       app.place(-120,100,0,180,0,180)


Summary:

The SICK PLOC 2D is an easy to use vision system that provides an intuitive user interface to setup vision tasks. With a simple TCP/IP communication interface that streams the responses in CSV format we can build an application class and get started. For PLC users, PLOC 2D supports EtherNet/IP which is also supported by Meca500 as well.


Please note that these examples are provided as-is by either Mecademic or it's Partners. These can be used as a starting point for development and testing but Mecademic or it's partners are not under obligation to provide support and are not liable for any errors or unintended behavior caused by the examples. These examples could be updated over time.

Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.