Machine Learning and Insurance

Machine learning applied to insurance data

For the insurance sector, we see machine learning as a fundamental game-changer since most insurance companies today are focused on three main objectives: improving compliance, improving cost structures and improving competitiveness. Machine learning can form at least part of the answer to all three.

Improving compliance: Today’s machine learning algorithms, techniques and technologies can be used to review, analyze and assess information in pictures, videos and voice conversations. One immediate benefit, for example, is the ability to better monitor and understand interactions between customers and sales agents in order to improve controls over mis-selling of products.

Improving cost structures: With a significant portion of an insurer’s cost structure devoted to human resources, any shift towards automation should deliver significant cost savings. Using machine learning insurers could cut their claims processing time down from a number of months to just a matter of minutes.

Improving competitiveness: While reduced cost structures and improved efficiency can lead to competitive advantage, there are many other ways that machine learning can give insurers the competitive edge, including product, service and process innovation.

Insurers overcome cultural differences to embrace artificial intelligence

Insurers have been slow to adopt machine learning in large part due to a culture of not being ‘early adopters’ of new technologies and approaches. This risk-averse culture also dampens the organization’s willingness to experiment and fail in its quest to uncover new approaches.

Insurance organizations also suffer from a cultural challenge common in information-intensive sectors: data hoarding. Fortunately, many companies are now keenly focused on moving towards a ‘data-driven’ culture that rewards information sharing and collaboration and discourages hoarding.

Starting small with machine learning

The first thing insurers should realize is that this is not an arms race. The winners will be the ones that take a measured and scientific approach to building up their machine learning capabilities and capacities and – over time – find new ways to incorporate machine learning into ever-more aspects of their business.

Insurers may want to start small. Our experience and research suggest that – given the cultural and risk challenges facing the insurance sector – insurers will want to start by developing a ‘proof of concept’ model that can safely be tested and adapted in a risk-free environment.

Recognizing that machines excel at routine tasks and that algorithms learn over time, insurers will want to focus their early ‘proof of concept’ efforts on those processes or assessments that are widely understood and add low value. The more decisions the machine makes and the more data it analyzes, the more prepared it will be to take on more complex tasks and decisions. Later, business leaders can start to think about developing the business case for industrialization, along with appropriate governance, monitoring and system management.

At KPMG, we have worked with a number of insurers to develop their ‘proof of concept’ machine learning strategies over the past year. We can say with absolute certainty that the battle of machines in the insurance sector has already started, and those that remain on the sidelines will suffer as they stand by and watch competitors find new ways to harness machines to drive increased efficiency and value.

Machine Learning & its impact on the future for Insurance

The interest in machine learning and the associated appetite to drive business outcomes from such investments continues to build. I’ve been talking to many insurance organisations over the past 18 months around machine learning and four consistent areas tend to arise as organisations grapple with the application and value of machine learning.

As 2017 gets well underway, I thought it prudent to share and gather opinion experiences in the insurance industry and I’ve also summarized these points of view as part of Louise Matthews’ ‘Five Minutes with….’ video series.

First and foremost, machine learning WILL change the way insurers do business. The insurance industry is founded on forecasting future events and estimating the value/impact of those events and has used established predictive modeling practices – especially in claims loss prediction and pricing – for some time now. With big data and new data sources such as sensors/telematics, external data sources (, digital (interactions), social and Web (sentiment), the opportunity to apply machine learning techniques has never been greater across new areas of insurance operations.

Machine Learning has now become an essential tool for insurers and it is used extensively across the core value chain to understand risk, claims and customer experience. Specifically, it is enabling insurance companies to yield higher predictive accuracy, as it can fit more flexible/complex models. As opposed to traditional statistical methods, machine learning takes advantage of the power of data analytics and is capable of computing seemingly unrelated datasets whether structured, semi-structured or unstructured.

By way of an example, predictive models based upon machine learning now take into consideration:

  • Structured data: type of loss, amount of loss, physician ID, etc.
  • Text: Notes, diaries, medical bills, accident reports, depositions, social data, invoices, etc.
  • Spatial, graph: accident location, work location, relationship of parties (physician, claimant, repair facilities), etc.
  • Time series: sequence of events/actions, claim date, accident date, duration between events/action, etc.

Now more than ever, insurers have the ability to evaluate mass amounts of underwriting/claims notes and diary (unstructured data), in addition to more standard documentation.

Pricing risk, estimating losses and monitoring fraud are critical areas that machine learning can support. Insurers have introduced machine learning algorithms primarily to handle risk similarity analytics, risk appetite and premium leakage. However, it is also widely used to aid the frequency/severity of claims, manage expenses, subrogation (general insurance), litigation and fraud.

One of the most impactful machine learning use cases is the ability to learn from audits of closed claims, as for the very first time leakage becomes controllable by the insurer. Claim audits are traditionally a manual process by nature, however, machine learning techniques provide an up-lift in the ability to learn from those by applying enhanced scoring and process methods throughout the claims lifecycle.

Those claim handling algorithms can be also used to monitoring and detecting fraud; however, one of the limiting factors may be the number of claims fraud cases/instances an insurance company has as the fraud datasets are fundamental for both traditional and machine learning models.

I’m often asked if machine learning can deliver a tangible decline in fraud rates and I do believe it can have an impact on earlier identification, or ‘counter-fraud’ techniques. The key element is to reduce the false positives and to apply machine learning algorithms to help determine which claims are potentially fraudulent vs. those that are legitimate.

Insurance companies applying this technique are reducing fraud in two aspects: earlier identification of the fraud and allocating resource time on the claim fraud investigation vs. spending on valid claims. This also increase customer satisfaction as valid claims are paid faster.

Nothing evidences the impact of any technology more than how it is applied in the real world and we are seeing those as relates to insurance fraud. Using machine learning, insurers can load claims data (whether structured, unstructured and semi-structured data) into a huge repository, often called “data lake”. This method differs from traditional predictive models which only leverage structured data. Claims notes, diaries and documents are key in discovering fraud and developing fraud models. In case of fraud detection, the procedure would consist on:

  • Learning Phase: where you are learning from “training data” or claims which are fraudulent and those which are valid. it consists on pre-processing (normalization, dimension reduction, image processing if you are using photos, aerial images etc), learning (supervised, unsupervised, minimization, etc.) and error analysis (precision, recall, overfitting, test/cross validation, etc.).
  • Prediction Phase: here one uses the model from the learning phase and apply it to new data and is deployed for detecting and flagging fraud.
  • Continuous Learning Phase: it is key to continuously recalibrate your models with new data and behaviors.

In addition to machine learning, the usage of Graph Analytics is also rapidly becoming popular because of its ability to visualise fraud patterns.

The usage of Graph Analytics with Apache Spark/GraphX is a newer method being leveraged as it enables the usage of neural network and social networks which is key in claims fraud analysis. This method is becoming quite popular vs. traditional claims scoring or business rules as these methods (considered a “flagging model”) and may result in too many false positives.

A Graph Analytics technique can help you understand the data relationships and is also used for investigating individual claims fraud cases. This method allows insurance companies to more quickly visualize fraud patterns vs. traditional scoring models.

Using machine learning for insurance pricing optimization

Wednesday, March 29, 2017

By Kaz Sato, Staff Developer Advocate, Google Cloud

AXA, the large global insurance company, has used machine learning in a POC to optimize pricing by predicting “large-loss” traffic accidents with 78% accuracy.

The TensorFlow machine-learning framework has been open source since just 2015, but in that relatively short time, its ecosystem has exploded in size, with more than 8,000 open source projects using its libraries to date. This increasing interest is also reflected by its growing role in all kinds of image-processing applications (with examples including skin cancer detection, diagnosis of diabetic eye disease and even sorting cucumbers), as well as natural-language processing ones such as language translation.

We’re also starting to see TensorFlow used to improve predictive data analytics for mainstream business use cases, such as price optimization. For example, in this post, I’ll describe why AXA, a large, global insurance company, built a POC using TensorFlow as a managed service on Google Cloud Machine Learning Engine for predicting “large-loss” car accidents involving its clients.

Understanding the use case

Approximately 7-10% of AXA’s customers cause a car accident every year. Most of them are small accidents involving insurance payments in the hundreds or thousands of dollars, but about 1% are so-called large-loss cases that require payouts over $10,000. As you might expect, it’s important for AXA adjusters to understand which clients are at higher risk for such cases in order to optimize the pricing of its policies.

Toward that goal, AXA’s R&D team in Japan has been researching the use of machine learning to predict if a driver may cause a large-loss case during the insurance period. Initially, the team had been focusing on a traditional machine-learning technique called Random Forest. Random Forest is a popular algorithm that uses multiple Decision Trees (such as possible reasons why a driver would cause a large-loss accident) for predictive modeling. Although Random Forest can be effective for certain applications, in AXA’s case, its prediction accuracy of less than 40% was inadequate.

In contrast, after developing an experimental deep learning (neural-network) model using TensorFlow via Cloud Machine Learning Engine, the team achieved 78% accuracy in its predictions. This improvement could give AXA a significant advantage for optimizing insurance cost and pricing, in addition to the possibility of creating new insurance services such as real-time pricing at point of sale. AXA is still at the early stages with this approach — architecting neural nets to make them transparent and easy to debug will take further development — but it’s a great demonstration of the promise of leveraging these breakthroughs.

How does it work?

AXA created a cool demo UI for the test bed. Let’s look at the details of its neural-network model that achieved the improvement.

AXA’s deep learning model demo UI

At the left side, you can see there are about 70 values as input features including the following.

  • Age range of the driver
  • Region of the driver’s address
  • Annual insurance premium range
  • Age range of the car

AXA entered these features into a single vector with 70 dimensions and put it into a deep learning model in the middle. The model is designed as a fully connected neural network with three hidden layers, with a ReLU as the activation function. AXA used data in Google Compute Engine to train the TensorFlow model, and Cloud Machine Learning Engine’s HyperTune feature to tune hyperparameters.

The following is the end result. The red line shows the accuracy rate with the deep learning model (78%).

Test results for POC

TensorFlow on business data

AXA’s case is one example of using machine learning for predictive analytics on business data. As another example, recently DeepMind used a machine-learning model to reduce the cost of Google data-center cooling by 40%. The team entered numerical values acquired from IoT sensors in Google data centers (temperatures, power, pump speeds, setpoints and so on) into a deep learning model and got better results than the existing approach.

Eclipse Download and Installation (for Python) Instructions

refer to:

Install Python and Java before installing Eclipse.

You may want to print these instructions before proceeding, so that you can refer to them while downloading and installing Eclipse. Or, just keep this document in your browser. You should read each step completely before performing the action that it describes. This document shows downloading and installing Eclipse on Windows 7 in Summer 2015.

Eclipse: Version 4.4 (Luna)

The Eclipse download requires about 200 MB of disk space; keep it on your machine, in case you need to re-install Eclipse. When installed, Eclipse requires an additional 200 MB of disk space.


  1. Click EclipseThe following page will appear in your browser. In this handout we will download Eclipse Standard 4.4 for Windows 32 Bit; if your computer uses Windows, continue below; otherwise look for the pull-down list showing Windows and instead choose either Mac OS X (Cocoa) or Linux and then continue below.

    It is critical that Java, Python, and Eclipse are either all 32 Bit or are all 64 Bit (and only if your Machine/OS supports 64 Bit): I think it easiest to use 32 Bit for everything.


  2. Click the Windows 32 Bit Operating System for your machine, under the heading Eclipse Standard 4.4 (right under Package Solutions).You will see the following page (don’t worry about the name of the institution to the right of the big downward pointing arrow).


  3. Click the orange DOWNLOAD button. The site named here, in orange to the right of the button: United States – Columbia University (http) is the random one chosen by the download page this time; yours may differ.This file should start downloading in your standard download folder, while showing a splash screen about donating to Eclipse. This file is about 200 Mb so it might take a while to download fully if you are on a slow internet connection (it took me about 5 minutes over a cable modem). Don’t worry about the exact time as long as the download continues to make steady progress. In Chrome progress is shown on the bottom-left of the window, via the icon

    The file should appear as

    Terminate the window browsing the Eclipse download.


  4. Move this file to a more permanent location, so that you can install Eclipse (and reinstall it later, if necessary).
  5. Start the Installing instructions directly below.

Installing (for Python)

Python and Java should be installed before installing Eclipse

  1. Unzip, the file that you just downloaded and moved.
    On my machine (running Windows 7), I can

    • Right-click the file.
    • Hover over the IZArc command from the menu of options.
    • Click Extract Here

    If you do not have IZArc or an equivalent unzipping program, here is the web site to download a free copy of IZarc.

    Unzipping this file creates a folder named eclipse; unzipping 200 MB can take a few minutes. You can leave this folder here or move it elsewhere on your hard disk. I recommend putting the downloaded file and resulting folder in the C:\Program Files\ directory.


  2. Create a shortcut on your desktop to the eclipse.exe file in this eclipse folder:
    On most Windows machines, you can

    • Right-press the file eclipse.exe
    • Drag it to the desktop.
    • Release the right button.
    • Click Create shortcut here

    Now you are ready to perform a one-time only setup of Eclipse on your machine.

  3. Double-click the shortcut to Eclipse that you just created above.If the following Security Warning window appears

    • Uncheck the Always ask before opening this file checkbox
    • Click Run

    The following splash screen will appear

    and then a Workspace Launcher pop-up window will appear.

    In the Workspace text box, your name should appear between C:\Users\ and \workspace, instead of Pattis.

    Leave unchecked the Use this as the default and do not ask again box. Although you will use this same workspace for the entire quarter (checking projects in and out of it), it is best to see this Workspace Launcher pop-up window each time you start Eclipse, to remind you where your workspace is located.

    In fact, it is a good idea to create on your desktop a shortcut to your workspace folder; but you must click OK (see below) before Eclipse creates this folder and you can create a shortcut to it.


  4. Click OK.Progress bars will appear as Eclipse loads.

    Eventually the Eclipse workbench will appear with a Welcome tab covering it.


  5. Terminate (click X on) the Welcome tab.You will not see the Welcome tab when you start Eclipse again, after this first time.
  6. Click Help (on the far right of the line below this window’s blue title Java – Eclipse) and then click Install New Software… in its pull-down menu, as shown below.

    The Install pop-up window will appear.


  7. Enter the text (or cut/paste from this document) into the Work with text box and press Enter (if that doesn’t work, try a few seconds the information under Name should change from
    There is no site selected to Pending to

    Select only the top PyDev checkbox; do NOT select the PyDev Mylyn Integration (optional) box.
    Check before proceeding that the Install pop-up window appears as


  8. Click Next >A new Install pop-up window will appear.


  9. Click Next >A new Install pop-up window will appear.

    Select the I accept the terms of the license agreement radio button, so the window appears as


  10. Click FinishAn Installing Software pop-up window will appear. During installation, it will show the various operations it is performing and a progress bar for each one. Because the installer is fetching various files online, this process might take a few minutes.

    The installation will pause, and a Selection Needed pop-up window appears.


  11. Check the box labeled Brainwy Software; Pydev; Brainwy.Check before proceeding that the Install pop-up window appears as


  12. Click OKThe installation will finish (showing the final progress bars).

    A Software Updates pop-up window will appear.


  13. Click YesEclipse will terminate and restart. As you did in step 3, click OK to the Workspace Launcher pop-up window.
  14. Click Window (to the left of Help on the far right of the line below this window’s blue title Java – Eclipse) and then click Preferences in its pull-down menu, as shown below.
  15. Drill down to select the interpreter:
    • Disclose PyDev by clicking the + in front of PyDev or double-clicking PyDev (after the +); the + changes to a .
    • Disclose Interpreters by clicking the + in front of Interpreters or double-clicking Interpreters (after the +); the + changes to a .
    • Select Python – Interpreter.

    The Preferences pop-up window should appear as follows

  16. Click New… at the top-right of the top windowA Select Interpreter pop-up window appears.

    Type Python34 in the Interpreter Name text box.
    Type (or cut/paste) C:\Python34\python.exe in the Interpreter Executable text box (or browse to this file).
    The Select Interpreter pop-up window should appear as


  17. Click OK.A Selection needed pop-up window will appear.

    Ensure that a check appears in all four checkboxes.


  18. Click OK.The Preferences pop-up window will be changed to


  19. Click OK.A Progress Information pop-up window will appear.

    During installation, it will show the various operations it is performing and a progress bar for each one. Eventually the installation will terminate.

    If the following pop-up window appears

    Ensure the “Private networks, such as my home or work network” checkbox is checked, and click Allow Access.


  20. Click Window (to the left of Help on the far right of the line below this window’s blue title Java – Eclipse SDK), hover over Open Perspective in the pull-down menu and then click Other… in the next pull-down menu.

    The Open Perspective pop-up window will appear.


  21. Double-click PyDev (third from the bottom)The blue label on the Eclipse window will change to PyDev – Eclipse, the Package Explorer in the upper-left tab will change to PyDev Package Explorer, and a PyDev icon will appear to the right of the Java icon.


  22. Right-click the Java icon to the left of the PyDev icon and select close

    The Eclipse workbench will now have only the PyDev icon here and will look like


  23. Locate the workspace folder created in step 3 (see C:\Users\yourname\workspace) and download the file into this workspace and unzip it there, producing the courselib folder; it should should contain a bunch of Python module files (ending in .py). From time to time you will be asked to put other course-specific Python module files into this courselib folder; these modules will be usable in all Python projects in Eclipse.
  24. Inform Eclipse to use the courselib folder in all Python projects.
    • Select Window | Preferences.
    • Disclose PyDev by clicking the + in front of PyDev or double-clicking PyDev (after the +); the + changes to a .
    • Disclose Interpreters by clicking the + in front of Interpreters or double-clicking Interpreters (after the +); the + changes to a .
    • Select Python – Interpreter.
    • Click the Libraries tab (leftmost, above the bottom pane)
    • Click the New Folder button (topmost, right of the bottom pane)
    • Browse to the courselib folder (added above) in the Browse for Folder pop-up window (C:\Users\yourname\workspace\courselib).
    • Click the courselib folder selected, and then click OK
    • Click Apply (you will see some progress bars)
      The <b<preferences< b=””> pop-up window should now appear as

      Notice a fifth entry (ending in courselib appears at the bottom of the bottom pane named System PYTHONPATH


    • Cick OK
  25. Terminate (click X on) the Eclipse window.

Eclipse is now installed for Python.


Import a web project to Eclipse

To make you feel easy to work around building your web project. I put my project in the sever as a war package. What you have to do is download this package and save it in your local folder (you have to remember where you stored) The package can be found in the following link:

MyWeb War

After you stored the package into your local folder, we can follow the steps below to import the web project into your eclipse.

  1. File–>Import ….
  2. Select–>Web–>WAR file
  3. Choose the package file you stored in your local folder for WAR file, fill a web project name (MyWebExample as what I used)
  4. We don’t have any extra lib used in our project, so leave it as an empty
  5. Finish everything, then we have a new web project, using all the existing my code

You are free to change anything in the project to play and learn HTML 5 and web.xml and servlets, or run this project on your personal Tomcat Server.



Deploy Java Web Apps to Azure using Eclipse

Refer to the article written by Senior Program Manger, Cloud and Enterprise

Today at EclipseCon 2016 Microsoft announced it joined the Eclipse foundation. This is an important next step for Microsoft to continue a meaningful conversation with the Java community, and especially the people who make tools based on Eclipse.

One of the first accomplishments as a new member of the foundation is the latest update to our Azure Toolkit for Eclipse. We’ve added Azure Web App support for Java applications and a few other enhancements based on feedback from the community.

Java Web Apps are fully managed so you can focus on building, deploying and scaling enterprise-grade apps in seconds.  Java Web apps run on either Tomcat or Jetty by default, but can be configured for other app servers as well. Here’s a tutorial on getting started with Java Web Apps.

Deploying to a Java Web App from Eclipse

You now have an option to deploy to a Java Web app running Tomcat or Jetty directly from Eclipse and unlike the more traditional Cloud Service deployments, the deployment takes seconds, not minutes. This is especially great when you want to deploy your Java code to a fully managed, scalable platform on  Azure with minimal configuration hassle.

Here’s an example showing deployment to a Java Web App using a JDK and web container provided by Azure.

In Eclipse, select the project you want to deploy to Azure and right-click to bring up the context menu.

In the context menu, select Azure, then click Publish to Azure Web App.

Next, you will be presented with options to choose: a name for the App Service, location, pricing tier and instance size.  Once those options are set, you can deploy your app. A few seconds later, your app is published, as shown here:

If you click on the Published link you will be taken to your running Java Web App.

You can explore installed apps via the built-in Azure explorer, also included when you install the Azure Toolkit for Eclipse:

This is a very simple example of a quick deployment using default options provide by Azure.  There are many more configuration and deployment options outlined in this tutorial.

Installing the Azure Toolkit for Eclipse

Refer to:

The Azure Toolkit for Eclipse provides templates and functionality that allow you to easily create, develop, test, and deploy Azure applications using the Eclipse development environment. The Azure Toolkit for Eclipse is an Open Source project, whose source code is available under the MIT License from the project’s site on GitHub at the following URL:

The following steps show you how to install the Azure Toolkit for Eclipse.


The Azure Toolkit for Eclipse requires the following software components:

  • Eclipse IDE for Java EE Developers, Mars, or later. You can download it at
  • A Java Developer Kit (JDK), version 1.8 or later.
  • An operating system. The Azure Toolkit for Eclipse has been tested on the following operating systems:
    • Windows 10, Windows 8 and 8.1, Windows 7
    • Windows Server 2012 and 2012 R2, Windows Server 2008 and 2008 R2
    • Mac OS X version “Yosemite” and later
    • Ubuntu Linux version 14, 15, and 16

For more information, see the Azure Toolkit for Eclipse page at the Eclipse Marketplace.

> If you are using the Azure Toolkit for Eclipse on Windows, the toolkit requires installing the Azure SDK 2.9.6 or later in order to use the Azure emulator. You have two options for installing the Azure SDK:
> * You can download and install the Azure SDK by using the [Web Platform Installer (WebPI)](
> * If you do not have the Azure SDK installed when you create your first Azure deployment project, you will be prompted to automatically download install the requisite version of the Azure SDK.
> Note that the Azure SDK is required on Windows only.

To install the Azure Toolkit for Eclipse

  1. Start Eclipse.
  2. When Eclipse opens, click the Help menu, and then click Install New Software, as shown in the following illustration.

    Installing the Azure Toolkit for Eclipse

  3. In the Available Software dialog, within the Work with text box, type followed by the Enter key.
  4. In the Name pane, check Azure Toolkit for Eclipse, and uncheck Contact all update sites during install to find required software. Your screen should appear similar to the following:

    Installing the Azure Toolkit for Eclipse

  5. If you expand the Azure Toolkit for Eclipse, you will see the following items:
    • Application Insights Plugin for Java: This component allows you to use Azure’s telemetry logging and analysis services for your applications and server instances.
    • Azure Access Control Services Filter: This component provides support for authenticating application users with Azure ACS, enabling single sign-on scenarios and externalizing identity logic from the application.
    • Azure Common Plugin: This component provides the common functionality needed by other toolkit components.
    • Azure Explorer for Eclipse: This component provides the common functionality needed by other toolkit components.
    • Azure Plugin for Eclipse with Java: This component provides support for developing projects that help build, test and deploy Java applications for the Microsoft Azure cloud in Eclipse and via command line.
    • Azure Web Apps Plugin with Java: This component provides support for deploying Java web applications to Microsoft Azure Web App containers.
    • Microsoft JDBC Driver 4.2 for SQL Server: This component provides JDBC API for SQL Server and Microsoft Azure SQL Database for Java Platform Enterprise Edition 8.
    • Package for Apache Qpid Client Libraries for JMS: This component provides the JMS client component from the Apache Qpid project to enable your application to use AMQP messaging in Microsoft Azure.
    • Package for Microsoft Azure Libraries for Java: This component provides APIs for accessing Microsoft Azure services, such as storage, service bus, service runtime, etc.
  6. Click Next. (If you experience unusual delays when installing the toolkit, ensure that Contact all update sites during install to find required software is unchecked.)
  7. In the Install Details dialog, click Next.

    Review Installation Details

  8. In the Review Licenses dialog, review the terms of the license agreements. If you accept the terms of the license agreements, click I accept the terms of the license agreements and then click Finish. (The remaining steps assume you do accept the terms of the license agreements. If you do not accept the terms of the license agreements, exit the installation process.)

    Review Licenses

    Eclipse will download and install the requisite packages.

    Installation Progress

  9. If prompted to restart Eclipse to complete installation, click Yes.

    Restart Prompt