This repository will not be updated. The repository will be kept available in read-only mode.
In this code pattern you will upload your own data into the Watson Discovery Service. Then you'll configure a web application so that it can query the data collection you created. The web app allows you to explore that data.
Once you have completed this code pattern, you will know how to:
- Build and run a Node.js API server with a HTML frontend written in React
- Configure Watson Discovery to build and enrich private data collections
- Use Watson Discovery to query and analyze data
- The cyber breach json files are added to the Discovery collection.
- The user interacts with the backend server via the app UI. The frontend app UI uses React to render search results and can reuse all of the views that are used by the backend for server side rendering. The frontend is using watson-react-components and is responsive.
- User input is processed and routed to the backend server, which is responsible for server side rendering of the views to be displayed on the browser. The backend server is written using express and uses express-react-views engine to render views written using React.
- The backend server sends user requests to the Watson Discovery Service. It acts as a proxy server, forwarding queries from the frontend to the Watson Discovery Service API while keeping sensitive API keys concealed from the user.
- Watson Discovery: A cognitive search and content analytics engine for applications to identify patterns, trends, and actionable insights.
- Node.js - An asynchronous event driven JavaScript runtime, designed to build scalable applications
- React - Javascript library for building User Interfaces
- Express - A popular and minimalistic web framework for creating API and Web server
Use the Deploy to IBM Cloud
button OR create the services and run locally.
-
Press the above
Deploy to IBM Cloud
button and then click onDeploy
. -
In Toolchains, click on Delivery Pipeline to watch while the app is deployed. Once deployed, the app can be viewed by clicking View app.
-
To see the app and services created and configured for this code pattern, use the IBM Cloud dashboard. The app is named
watson-data-breaches
with a unique suffix. The following services are created:- breaches-discovery-service
NOTE: These steps are only needed when running locally instead of using the
Deploy to IBM Cloud
button.
- Clone the repo
- Create IBM Cloud services
- Load the Discovery files
- Configure credentials
- Run the application
Clone the watson-discovery-analyze-data-breaches
repo locally. In a terminal, run:
git clone https://github.com/ibm/watson-discovery-analyze-data-breaches
We'll be using the folder data/breaches/
Create the following services:
Launch the Watson Discovery tool. Create a new data collection and give the data collection a unique name.
Save the environment_id and collection_id for your
.env
file in the next step.
Under Add data to this collection
use Drag and drop your documents here or browse from computer
to seed the content with the json files in data/breaches/
.
The credentials for the IBM Cloud Discovery service can be found in the Services
menu in IBM Cloud,
by selecting the Service Credentials
option for the service.
The other settings for Discovery were collected during the earlier setup steps (DISCOVERY_COLLECTION_ID
and DISCOVERY_ENVIRONMENT_ID
).
Copy the env.example
to .env
.
cp env.example .env
Edit the .env
file with the necessary settings.
# Copy this file to .env and replace the credentials with
# your own before starting the app.
# Watson Discovery
DISCOVERY_URL=<add_discovery_url>
DISCOVERY_ENVIRONMENT_ID=<add_discovery_environment_id>
DISCOVERY_COLLECTION_ID=<add_discovery_collection_id>
## Un-comment and use either username+password or IAM apikey.
# DISCOVERY_USERNAME=<add_discovery_username>
# DISCOVERY_PASSWORD=<add_discovery_password>
# DISCOVERY_IAM_APIKEY=<add_discovery_iam_apikey>
# Run locally on a non-default port (default is 3000)
# PORT=3000
- Install Node.js runtime or NPM.
- Start the app by running
npm install
, followed bynpm start
. - Use the chatbot at
localhost:3000
.
Note: server host can be changed as required in server.js and
PORT
can be set in.env
.
-
Error: Environment {GUID} is still not active, retry once status is active
This is common during the first run. The app tries to start before the Discovery environment is fully created. Allow a minute or two to pass. The environment should be usable on restart. If you used
Deploy to IBM Cloud
button the restart should be automatic. -
Error: Only one free environent is allowed per organization
> To work with a free trial, a small free Discovery environment is created. If you already have a Discovery environment, this will fail. If you are not using Discovery, check for an old service thay you may want to delete. Otherwise use the .env DISCOVERY_ENVIRONMENT_ID
to tell the app which environment you want it to use. A collection will be created in this environment using the default configuration.
- Demo on Youtube: Watch the video.
- Watson Node.js SDK: Download the Watson Node SDK.
- Blog: Bring your own data to Watson Discovery Service: Steps to building the data for this code pattern.
- Data set: World’s biggest data breaches (Information is Beautiful): Check out the data set used for this code pattern, as well as others available for you to use.
- Watson Discovery, an IBM API adding value to corporate data: Dive into IBM Watson Discovery Service, enabling companies to structure and understand large masses of data.
- Blog: Watson Discovery Service – Understand your data at scale with less effort: Align and connect different data sets to expose critical correlations and causal factors.
- Blog: Using IBM Watson Discovery to query unstructured data: Make sense of and identify patterns in large amounts of unstructured data.
- Artificial Intelligence Code Patterns: Enjoyed this code pattern? Check out our other AI Code Patterns.
- AI and Data Code Pattern Playlist: Bookmark our playlist with all of our code pattern videos
- With Watson: Want to take your Watson app to the next level? Looking to utilize Watson Brand assets? Join the With Watson program to leverage exclusive brand, marketing, and tech resources to amplify and accelerate your Watson embedded commercial solution.
This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache License, Version 2.