20 - Smart Software for Autonmous Maritime Systems

Submitting Institutions

Heriot-Watt University,
University of Edinburgh

Unit of Assessment

General Engineering

Summary Impact Type

Technological

Research Subject Area(s)

Information and Computing Sciences: Artificial Intelligence and Image Processing, Information Systems
Engineering: Electrical and Electronic Engineering


Download original

PDF

Summary of the impact

Strong collaboration and associated technology transfer from ERPE have enabled SeeByte to stay at the forefront of technology, securing strategic partnerships including Subsea7, BAE SYSTEMS and the US Navy in the offshore and military markets. This has enabled sustained employment in the science and engineering sector growing to 50 staff and financial growth, 15 technology licenses from ERPE have directly or indirectly generated £11 million in revenues for SeeByte in the REF impact period. In October 2013 SeeByte was acquired by Bluefin Robotics Inc, a spin out of MIT owned by the Battelle group [text removed for publication].

Underpinning research

ERPE staff throughout the REF period(Prof Lane, Prof Petillot, Dr Brown (Senior Lecturer), Dr Clark (Reader)) have been and continue to be active in underwater robotics and sensing research, pursuing fundamental and applied research in autonomous systems, control, data processing (automatic target recognition, simultaneous localisation and mapping (SLAM), mosaicing, visual servoing) and sonar design (wideband bio-sonar). Of these, Lane and Petillot were directly involved in research relevant to SeeByte business and its subsequent technology transfer [S8].

Overall, the most important outputs from the ERPE research which have enabled SeeByte to establish new products were:

  • The development of algorithms for tracking and identification of structures in sonar and video as well as visual servoing [1] were instrumental to the success of these projects and were adapted and productised by SeeByte. This work was carried out under the Alive project[G1] and Autotracker project [G2]where our advances were developed and applied to autonomous docking on subsea platforms (the world's first in 2003) and pipeline inspection using an Autonomous Underwater Vehicle (AUV).
  • The development of new algorithmic techniques for sonar interpretation, fusion and mosaicing, enabled the building of consistent maps of the subsea environment [2,3]. The algorithms were subsequently licenced to SeeByte Ltd and became an integral part of its SeeTrack product as well as several US Navy projects. The Performance Analysis and Training Tool in SeeTrack directly used [3] and is now sold to the Netherlands, Canadian, US and UK navies. This work was carried out under the Amason project[G3] and Consensus project[G4] and EPSRC-funded PhD studentship (Reed 1998-2001).
  • Research in multiple vehicle collaboration technologies including planning [4], communications and world representation [5] as well as fault diagnostics. New techniques for multiple vehicle collaboration in terms of plan execution where goals are shared across the collective and in a low bandwidth and unreliable communication context were also developed [5]. This work was carried out under the MoD and EU funding [G5], [G6],[G7],[G8] and EPSRC funded studentship (Hamilton 1998-2001).

References to the research

The references identified with * are the ones which best indicate the quality of the underpinning research.

[1] J.Evans, B. Smith, P. Patron, D.M. Lane "Design and evaluation of a reactive and deliberative collision avoidance and escape architecture for autonomous robots" Autonomous Robots April 2008, Volume 24/3, pp 247-266 DOI:10.1007/s10514-007-9053-8 18 Google Scholar (GS) citations.
Core technology in SeeTrack CoPilot and the Autonomous Inspection Vehicle.

 
 
 
 

[2] * E. Coiras, Y.R. Petillot, D.M.Lane "Multi-Resolution 3D Reconstruction from Side-Scan Sonar Images" IEEE Transactions on Image Processing Vol. 16, No 2, pp 382-390 (2007) DOI:10.1109/TIP.2006.88833718 GS citations.
This paper developed an algorithm for augmented reality in sonar imagery that was the foundation for the Performance Analysis and Training Tool PATT developed in SeeByte, part of the SeeTrack Military product sold widely in the military market.

 
 
 
 

[3] * S. Reed, Y. Petillot, J. Bell. "An automated approach to the detection and extraction of mine features in side-scan sonar". IEEE Journal Oceanic Engineering, Vol. 28, pp 90-105, Jan 2003. DOI:10.1109/JOE.2002.808199123 GS citations.
Basis for SeeByte Automatic Target Recognition product now sold as part of their SeeTrack Military software suite to 12 Navies worldwide.

 
 
 
 

[4] C.C. Sotzing, D.M.Lane "Improving the Co-ordination Efficiency of Limited Communication Multi-AUV Operations Using a Multi-Agent Architecture" Journal of Field Robotics Vol 27 No 4 July 2010 pp.412-429 DOI: 10.1002/rob.2034011 GS citations.
This paper forms the basis of the Mission Executive in the Autonomous Inspection Vehicle and SeeTrack Neptune products in SeeByte.

 
 
 
 

[5] * E. Miguelañez, P. Patrón, K.E.Brown, Y.R. Petillot, D.M. Lane "Semantic Knowledge-Based Framework to Improve the Situation Awareness of Autonomous Underwater Vehicles", IEEE Transactions on Knowledge and Data Engineering, vol.23, no.5, pp.759-773, May 2011 http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.46 . 13 GS citations.
This paper forms the basis of the world modelling in the SeeTrack Neptune product in SeeByte (2013), now sold to the US, UK and Canada.

 
 
 
 

Grants

[G1] EU FP5, G2RD-CT-2000-00269, €2.4M, Cybernetix S.A. (PI), ERPE and 3 other partners,"Autonomous light intervention vehicle: LIVE", 2001-2003.

[G2] EU FP5, G3RD-CT-2000-00265, €1.8M, ERPE (PI) and 5 other partners, "Autonomous inspection of subsea telecommunications cables, power cables and pipelines: AUTOTRACKER",2001-2004.

[G3] EU FP5, EVK3-CT-2001-00059, €1.7M, ERPE (PI) and 6 other partners,"Advanced mapping with sonar and video: AMASON", 2002-2004.

[G4] EPSRC, GR/S16980/01, £129k (Pre-FEC), Y. Petillot, "CONcurrent Hierarchial SENSor Registration and FUSion (CONSENSUS): Applications to seabed classification", 2003-2007.

[G5] Ministry of Defence, BAUUV-POR002602, £150k, "Battlespace Access Unmanned Underwater Vehicle (BAUUV)",2003-06,

[G6] Ministry of Defence, BAES-FOAS-PRC-CT-0811, £70M, BAE Systems (PI), ERPE, 21 industry and 19 other university partners, "Systems Engineering for Autonomous Systems Defence Technology Centre: SEAS DTC", 2005-2012.

[G7] Ministry of Defence, ERPE contract no.RT/COM/5/058, £370k"Competition of Ideas", 2007-2010.

[G8] EU FP6, 12526, €19.9M, Union of European Railway Industries (PI), ERPE and 37 other partners, "INTElligent integration of RAILway systems: INTEGRAIL", 2005-2009.

Details of the impact

The research outputs of ERPE have directly contributed to establishing SeeByte's (http://www.seebyte.com/latest-news/) leadership in autonomy software in the maritime domain. Over the period 2008-2014, 15 technology licences from ERPE have generated £11 million in revenues for SeeByte [S1].

SeeByte's SeeTrack Software is the de-facto standard for AUV mission planning, real-time data processing and post mission analysis in the military and increasingly the offshore domain for maritime unmanned systems. SeeTrack Military is currently sold to 15 Navies worldwide, including those of the US [S4], UK [S3], Canada, Netherlands and New Zealand. It provides full situational awareness and additional tools such as Automatic Target Recognition. Based on the research reported in [4], and a performance training tool based on the work in [5][S2], licence sales for these products amount to £1.7M for SeeByte Ltd over the last 5 years [S1].

"Professors Lane & Petillot and their University colleagues have contributed great value to the United States Office of Naval Research Mine Warfare Science and Technology programs through their sharing of innovative technology concepts with SeeByte Ltd., who in turn have provided and continue to provide prototype advanced software systems to the ONR programs. SeeByte have played key roles in our Future Naval Capability developments by providing capable algorithms for our autonomous underwater vehicle efforts. These have included algorithms for tracking and identification of structures in sonar and video, sensor-reactive manoeuvring capability for autonomous vehicle operations in complex domains and mission planning and analysis software for missions involving multiple heterogeneous unmanned systems. Their Performance Analysis and Training Tool (PATT), which had its roots at HWU, is a key element of our algorithm training scheme. ONR system development and demonstration has resulted in several systems now in service in the United States Navy. Many of these now-operational assets can trace their fundamental origin to innovative research carried out at ERPE", Head of Mine Warfare Science and Technology US Office of Naval Research [S4]

SeeTrack Neptune System [4, 5] is software enabling multiple vehicle collaboration and planning for mine counter measures using Autonomous Underwater and Surface Vehicles. It was recently licenced to the UK MoD [S3] and the Canadian Defence organisation DRDC, generating £1.6M of licence, support and engineering sales for SeeByte Ltd.

"SeeByte Ltd has benefited enormously over the years from being able to access University research outputs and turn them into products that make a difference in the real world. The technology transfer was facilitated by the availability of highly trained graduates as well as intellectual property. HWU original research sits at the core of some of our flagship products, from multiple vehicle collaboration (SeeTrack Neptune) to Sonar sensor processing (SeeTrack PATT) and autonomous systems mission planning, monitoring and data analysis (SeeTrack Military). Our collaboration remains strong and active. We expect that to continue under the new SeeByte ownership, as demonstrated by our support for Professor Petillot's current Royal Society Industry Fellowship in SeeByte Ltd", SeeByte CEO [S1]

SeeTrack Co-Pilot productbuilds on the research reported in [1] and is now the software of choice for smart ROVs. This work has led to strategic partnerships being secured by SeeByte Ltd with Soil Machine Dynamics (SMD) for development of large offshore ROVs, and with VideoRay for small inspection ROVs. Both companies are market leaders in their respective sectors.

The SeeTrack Autotracker product is a system based on the research outlined in [1] above. It is the only commercially available pipeline tracking software enabling autonomous underwater vehicles to inspect pipelines and generate high resolution video and sonar data to verify asset integrity. It has been sold to 5 customers, generating £480k of revenue in the REF2014 period [S1].

The SeeByte Autotracker product holds the world record for autonomous pipeline inspection [S6] and is sold worldwide. The SeeTrack Co-Pilot product is now the standard software for SmartROV autopilots. SeeByte's Autonomous Inspection Vehicle programme (AIV) (operated by Subsea 7) offers the world's first commercial service for autonomous inspection of deep-water oilfields. All of these technologies are also an integral part of the Autonomous Inspection Vehicle, a multi-million dollar programme in collaboration with Subsea7, which has provided the first commercial system for autonomous inspection of deep-water oilfield infrastructure, in service from Q4 2013 [S5][S7].http://www.rovworld.com/article4343.html

In October 2013, SeeByte was acquired by Bluefin Robotics Inc. [text removed for publication], with the SeeByte brand to be retained [S10].The combined companies offer a complete suite of autonomy products for surface vehicles and underwater vehicles, both remotely-operated and unmanned.

Sources to corroborate the impact

[S1] SeeByte Commercial: (Senior Operating Officer) will be able to confirm the economic performance of SeeByte, who key markets are and how the research informed the business development of the company

[S2] SeeByte Technical: (Senior Engineer) will be able to describe the applications of the technology by worldwide customers and how ERPE research has informed product development

[S3] UK MoD procurements: Business Development Manager, DSTL Naval Systems.Will corroborate the licensing of the NEPTUNE System by the UK MOD and will confirm that the UK Navy bought the SeeTrack software.

[S4] Head of Mine Warfare Science and Technology, Office of Naval Research. Seebyte's SeeTrack Software is now the de-facto standard for AUV mission planning, data processing and post mission analysis in the military and increasingly the offshore domain for maritime unmanned systems. Will confirm that the US Navy has bought the software.

[S5] Engineering Manager (Remote Systems), SubSea7. Will confirm that Subsea7 are collaborators on the AIV programme; a multi-million Autonomous Inspection Vehicle programme which provides the first commercial autonomous inspection system in the offshore market, launched in Q4 2013

Further public links on SeeByte business impacts:

[S6] Autotracker world record:http://www.unmanned.co.uk/unmanned-vehicles-news/unmanned-autonomous-underwater-vehicles-uuv-auv-news/seebyte-breaks-record-with-gavia-auv/

[S7] AIV announcement and video:http://subseaworldnews.com/2013/03/20/video-subsea-7s-autonomous-inspection-vehichle-aiv/

[S8] Praxis-Unico Award: http://www.impactawards.org.uk/uploads/SeeByte%20v2.pdf

[S9] Scottish Digital Technology Award:http://www.hydro-international.com/news/id6282-Award_for_International_Growth_for_SeeByte.html

[S10] Press release confirming Bluefin Robotics' acquisition of SeeByte
http://www.bluefinrobotics.com/news-and-downloads/press/bluefin-acquires-seebyte-adding-new-software-capabilities-to-maritime-robotics-portfolio/