20 - Smart Software for Autonmous Maritime Systems
Submitting Institutions
Heriot-Watt University,
University of EdinburghUnit of Assessment
General EngineeringSummary Impact Type
TechnologicalResearch Subject Area(s)
Information and Computing Sciences: Artificial Intelligence and Image Processing, Information Systems
Engineering: Electrical and Electronic Engineering
Summary of the impact
Strong collaboration and associated technology transfer from ERPE have
enabled SeeByte to stay at the forefront of technology, securing strategic
partnerships including Subsea7, BAE SYSTEMS and the US Navy in the
offshore and military markets. This has enabled sustained employment in
the science and engineering sector growing to 50 staff and financial
growth, 15 technology licenses from ERPE have directly or indirectly
generated £11 million in revenues for SeeByte in the REF impact period. In
October 2013 SeeByte was acquired by Bluefin Robotics Inc, a spin out of
MIT owned by the Battelle group [text removed for publication].
Underpinning research
ERPE staff throughout the REF period(Prof Lane, Prof Petillot, Dr Brown
(Senior Lecturer), Dr Clark (Reader)) have been and continue to be active
in underwater robotics and sensing research, pursuing fundamental and
applied research in autonomous systems, control, data processing
(automatic target recognition, simultaneous localisation and mapping
(SLAM), mosaicing, visual servoing) and sonar design (wideband bio-sonar).
Of these, Lane and Petillot were directly involved in research relevant to
SeeByte business and its subsequent technology transfer [S8].
Overall, the most important outputs from the ERPE research which have
enabled SeeByte to establish new products were:
- The development of algorithms for tracking and identification of
structures in sonar and video as well as visual servoing [1] were
instrumental to the success of these projects and were adapted and
productised by SeeByte. This work was carried out under the Alive
project[G1] and Autotracker project [G2]where our advances were
developed and applied to autonomous docking on subsea platforms (the
world's first in 2003) and pipeline inspection using an Autonomous
Underwater Vehicle (AUV).
- The development of new algorithmic techniques for sonar
interpretation, fusion and mosaicing, enabled the building of consistent
maps of the subsea environment [2,3]. The algorithms were subsequently
licenced to SeeByte Ltd and became an integral part of its SeeTrack
product as well as several US Navy projects. The Performance Analysis
and Training Tool in SeeTrack directly used [3] and is now sold to the
Netherlands, Canadian, US and UK navies. This work was carried out under
the Amason project[G3] and Consensus project[G4] and EPSRC-funded PhD
studentship (Reed 1998-2001).
- Research in multiple vehicle collaboration technologies including
planning [4], communications and world representation [5] as well as
fault diagnostics. New techniques for multiple vehicle collaboration in
terms of plan execution where goals are shared across the collective and
in a low bandwidth and unreliable communication context were also
developed [5]. This work was carried out under the MoD and EU funding
[G5], [G6],[G7],[G8] and EPSRC funded studentship (Hamilton 1998-2001).
References to the research
The references identified with * are the ones which best indicate the
quality of the underpinning research.
[1] J.Evans, B. Smith, P. Patron, D.M. Lane "Design and evaluation of a
reactive and deliberative collision avoidance and escape architecture for
autonomous robots" Autonomous Robots April 2008, Volume 24/3, pp 247-266
DOI:10.1007/s10514-007-9053-8
18 Google Scholar (GS) citations.
Core technology in SeeTrack CoPilot and the Autonomous Inspection Vehicle.
[2] * E. Coiras, Y.R. Petillot, D.M.Lane "Multi-Resolution 3D
Reconstruction from Side-Scan Sonar Images" IEEE Transactions on Image
Processing Vol. 16, No 2, pp 382-390 (2007) DOI:10.1109/TIP.2006.88833718
GS citations.
This paper developed an algorithm for augmented reality in sonar imagery
that was the foundation for the Performance Analysis and Training Tool
PATT developed in SeeByte, part of the SeeTrack Military product sold
widely in the military market.
[3] * S. Reed, Y. Petillot, J. Bell. "An automated approach to the
detection and extraction of mine features in side-scan sonar". IEEE
Journal Oceanic Engineering, Vol. 28, pp 90-105, Jan 2003. DOI:10.1109/JOE.2002.808199123
GS citations.
Basis for SeeByte Automatic Target Recognition product now sold as part of
their SeeTrack Military software suite to 12 Navies worldwide.
[4] C.C. Sotzing, D.M.Lane "Improving the Co-ordination Efficiency of
Limited Communication Multi-AUV Operations Using a Multi-Agent
Architecture" Journal of Field Robotics Vol 27 No 4 July 2010 pp.412-429
DOI: 10.1002/rob.2034011
GS citations.
This paper forms the basis of the Mission Executive in the Autonomous
Inspection Vehicle and SeeTrack Neptune products in SeeByte.
[5] * E. Miguelañez, P. Patrón, K.E.Brown, Y.R. Petillot, D.M. Lane
"Semantic Knowledge-Based Framework to Improve the Situation Awareness of
Autonomous Underwater Vehicles", IEEE Transactions on Knowledge and Data
Engineering, vol.23, no.5, pp.759-773, May 2011 http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.46
. 13 GS citations.
This paper forms the basis of the world modelling in the SeeTrack Neptune
product in SeeByte (2013), now sold to the US, UK and Canada.
Grants
[G1] EU FP5, G2RD-CT-2000-00269, €2.4M, Cybernetix S.A. (PI), ERPE and 3
other partners,"Autonomous light intervention vehicle: LIVE", 2001-2003.
[G2] EU FP5, G3RD-CT-2000-00265, €1.8M, ERPE (PI) and 5 other partners,
"Autonomous inspection of subsea telecommunications cables, power cables
and pipelines: AUTOTRACKER",2001-2004.
[G3] EU FP5, EVK3-CT-2001-00059, €1.7M, ERPE (PI) and 6 other
partners,"Advanced mapping with sonar and video: AMASON", 2002-2004.
[G4] EPSRC, GR/S16980/01, £129k (Pre-FEC), Y. Petillot, "CONcurrent
Hierarchial SENSor Registration and FUSion (CONSENSUS): Applications to
seabed classification", 2003-2007.
[G5] Ministry of Defence, BAUUV-POR002602, £150k, "Battlespace Access
Unmanned Underwater Vehicle (BAUUV)",2003-06,
[G6] Ministry of Defence, BAES-FOAS-PRC-CT-0811, £70M, BAE Systems (PI),
ERPE, 21 industry and 19 other university partners, "Systems Engineering
for Autonomous Systems Defence Technology Centre: SEAS DTC", 2005-2012.
[G7] Ministry of Defence, ERPE contract no.RT/COM/5/058,
£370k"Competition of Ideas", 2007-2010.
[G8] EU FP6, 12526, €19.9M, Union of European Railway Industries (PI),
ERPE and 37 other partners, "INTElligent integration of RAILway systems:
INTEGRAIL", 2005-2009.
Details of the impact
The research outputs of ERPE have directly contributed to establishing
SeeByte's (http://www.seebyte.com/latest-news/)
leadership in autonomy software in the maritime domain. Over the period
2008-2014, 15 technology licences from ERPE have generated £11 million in
revenues for SeeByte [S1].
SeeByte's SeeTrack Software is the de-facto standard for AUV mission
planning, real-time data processing and post mission analysis in the
military and increasingly the offshore domain for maritime unmanned
systems. SeeTrack Military is currently sold to 15 Navies worldwide,
including those of the US [S4], UK [S3], Canada, Netherlands and New
Zealand. It provides full situational awareness and additional tools such
as Automatic Target Recognition. Based on the research reported in [4],
and a performance training tool based on the work in [5][S2], licence
sales for these products amount to £1.7M for SeeByte Ltd over the last 5
years [S1].
"Professors Lane & Petillot and their University colleagues have
contributed great value to the United States Office of Naval Research
Mine Warfare Science and Technology programs through their sharing of
innovative technology concepts with SeeByte Ltd., who in turn have
provided and continue to provide prototype advanced software systems to
the ONR programs. SeeByte have played key roles in our Future Naval
Capability developments by providing capable algorithms for our
autonomous underwater vehicle efforts. These have included algorithms
for tracking and identification of structures in sonar and video,
sensor-reactive manoeuvring capability for autonomous vehicle operations
in complex domains and mission planning and analysis software for
missions involving multiple heterogeneous unmanned systems. Their
Performance Analysis and Training Tool (PATT), which had its roots at
HWU, is a key element of our algorithm training scheme. ONR system
development and demonstration has resulted in several systems now in
service in the United States Navy. Many of these now-operational assets
can trace their fundamental origin to innovative research carried out at
ERPE", Head of Mine Warfare Science and Technology US Office of
Naval Research [S4]
SeeTrack Neptune System [4, 5] is software enabling multiple vehicle
collaboration and planning for mine counter measures using Autonomous
Underwater and Surface Vehicles. It was recently licenced to the UK MoD
[S3] and the Canadian Defence organisation DRDC, generating £1.6M of
licence, support and engineering sales for SeeByte Ltd.
"SeeByte Ltd has benefited enormously over the years from being able
to access University research outputs and turn them into products that
make a difference in the real world. The technology transfer was
facilitated by the availability of highly trained graduates as well as
intellectual property. HWU original research sits at the core of some of
our flagship products, from multiple vehicle collaboration (SeeTrack
Neptune) to Sonar sensor processing (SeeTrack PATT) and autonomous
systems mission planning, monitoring and data analysis (SeeTrack
Military). Our collaboration remains strong and active. We expect that
to continue under the new SeeByte ownership, as demonstrated by our
support for Professor Petillot's current Royal Society Industry
Fellowship in SeeByte Ltd", SeeByte CEO [S1]
SeeTrack Co-Pilot productbuilds on the research reported in [1] and is
now the software of choice for smart ROVs. This work has led to strategic
partnerships being secured by SeeByte Ltd with Soil Machine Dynamics (SMD)
for development of large offshore ROVs, and with VideoRay for small
inspection ROVs. Both companies are market leaders in their respective
sectors.
The SeeTrack Autotracker product is a system based on the research
outlined in [1] above. It is the only commercially available pipeline
tracking software enabling autonomous underwater vehicles to inspect
pipelines and generate high resolution video and sonar data to verify
asset integrity. It has been sold to 5 customers, generating £480k of
revenue in the REF2014 period [S1].
The SeeByte Autotracker product holds the world record for autonomous
pipeline inspection [S6] and is sold worldwide. The SeeTrack Co-Pilot
product is now the standard software for SmartROV autopilots. SeeByte's
Autonomous Inspection Vehicle programme (AIV) (operated by Subsea 7)
offers the world's first commercial service for autonomous inspection of
deep-water oilfields. All of these technologies are also an integral part
of the Autonomous Inspection Vehicle, a multi-million dollar programme in
collaboration with Subsea7, which has provided the first commercial system
for autonomous inspection of deep-water oilfield infrastructure, in
service from Q4 2013 [S5][S7].http://www.rovworld.com/article4343.html
In October 2013, SeeByte was acquired by Bluefin Robotics Inc. [text removed
for publication], with the SeeByte brand to be retained [S10].The combined
companies offer a complete suite of autonomy products for surface vehicles
and underwater vehicles, both remotely-operated and unmanned.
Sources to corroborate the impact
[S1] SeeByte Commercial: (Senior Operating Officer) will be able to
confirm the economic performance of SeeByte, who key markets are and how
the research informed the business development of the company
[S2] SeeByte Technical: (Senior Engineer) will be able to describe the
applications of the technology by worldwide customers and how ERPE
research has informed product development
[S3] UK MoD procurements: Business Development Manager, DSTL Naval
Systems.Will corroborate the licensing of the NEPTUNE System by the UK MOD
and will confirm that the UK Navy bought the SeeTrack software.
[S4] Head of Mine Warfare Science and Technology, Office of Naval
Research. Seebyte's SeeTrack Software is now the de-facto standard for AUV
mission planning, data processing and post mission analysis in the
military and increasingly the offshore domain for maritime unmanned
systems. Will confirm that the US Navy has bought the software.
[S5] Engineering Manager (Remote Systems), SubSea7. Will confirm that
Subsea7 are collaborators on the AIV programme; a multi-million Autonomous
Inspection Vehicle programme which provides the first commercial
autonomous inspection system in the offshore market, launched in Q4 2013
Further public links on SeeByte business impacts:
[S6] Autotracker world record:http://www.unmanned.co.uk/unmanned-vehicles-news/unmanned-autonomous-underwater-vehicles-uuv-auv-news/seebyte-breaks-record-with-gavia-auv/
[S7] AIV announcement and video:http://subseaworldnews.com/2013/03/20/video-subsea-7s-autonomous-inspection-vehichle-aiv/
[S8] Praxis-Unico Award: http://www.impactawards.org.uk/uploads/SeeByte%20v2.pdf
[S9] Scottish Digital Technology Award:http://www.hydro-international.com/news/id6282-Award_for_International_Growth_for_SeeByte.html
[S10] Press release confirming Bluefin Robotics' acquisition of SeeByte
http://www.bluefinrobotics.com/news-and-downloads/press/bluefin-acquires-seebyte-adding-new-software-capabilities-to-maritime-robotics-portfolio/