Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add document for node conformance test. #1216

Merged
merged 1 commit into from
Sep 15, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions _data/guides.yml
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,8 @@ toc:
path: /docs/getting-started-guides/ubuntu/
- title: Ubuntu Nodes with Calico
path: /docs/getting-started-guides/ubuntu-calico/
- title: Validate Node Setup
path: /docs/admin/node-conformance
- title: Portable Multi-Node Cluster
path: /docs/getting-started-guides/docker-multinode/
- title: Building Large Clusters
Expand Down
114 changes: 114 additions & 0 deletions docs/admin/node-conformance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
assignees:
- Random-Liu

---

* TOC
{:toc}

## Node Conformance Test

*Node conformance test* is a test framework validating whether a node meets the
minimum requirement of Kubernetes with a set of system verification and
functionality test. A node which passes the tests is qualified to join a
Kubernetes cluster.

## Limitations

There are following limitations in the current implementation of node
conformance test. They'll be improved in future version.

* Node conformance test only supports Docker as the container runtime.
* Node conformance test doesn't validate network related system configurations
and functionalities.

## Prerequisite

Node conformance test is used to test whether a node is ready to join a
Kubernetes cluster, so the prerequisite is the same with a standard Kubernetes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any documentation on the "prerequisite" of a standard kubernetes npde?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to link somewhere, but I don't think we have it. :(

node. At least, the node should have properly installed:

* Container Runtime (Docker)
* Kubelet

Node conformance test validates kernel configurations. If the kenrel module
`configs` is built as module in your environment, it must be loaded before the
test. (See [Caveats #3](#caveats) for more information)

## Usage

### Run Node Conformance Test

* **Step 1:** Point your Kubelet to localhost `--api-servers="http://localhost:8080"`,
because the test framework starts a local master to test Kubelet.

* **Step 2:** Run the node conformance test with command:

```shell
# $CONFIG_DIR is the pod manifest path of your kubelet.
# $LOG_DIR is the test output path.
sudo docker run -it --rm --privileged --net=host \
-v /:/rootfs:ro -v /var/run:/var/run \
-v $CONFIG_DIR:/etc/manifest -v $LOG_DIR:/var/result \
gcr.io/google_containers/node-test-amd64:v0.1
```

### Run Node Conformance Test for Other Architectures

We also build node conformance test docker images for other architectures:

Arch | Image |
--------|:-----------------:|
amd64 | node-test-amd64 |
arm | node-test-arm |
arm64 | node-test-arm64 |

### Run Selected Test

In fact, Node conformance test is a containerized version of [node e2e
test](https://github.com/kubernetes/kubernetes/blob/release-1.4/docs/devel/e2e-node-tests.md).
By default, it runs all conformance test.

Theoretically, you can run any node e2e test if you configure the container and
mount required volumes properly. But **it is strongly recommended to only run conformance
test**, because the non-conformance test needs much more complex framework configuration.

To run specific tests, overwrite the environment variable `FOCUS` with the
regular expression of tests you want to run.

```shell
sudo docker run -it --rm --privileged --net=host \
-v /:/rootfs:ro -v /var/run:/var/run \
-v $CONFIG_DIR:/etc/manifest -v $LOG_DIR:/var/result \
-e FOCUS=MirrorPod \ # Only run MirrorPod test
gcr.io/google_containers/node-test-amd64:v0.1
```

To skip specific tests, overwrite the environment variable `SKIP` with the
regular expression of tests you want to skip.

```shell
sudo docker run -it --rm --privileged --net=host \
-v /:/rootfs:ro -v /var/run:/var/run \
-v $CONFIG_DIR:/etc/manifest -v $LOG_DIR:/var/result \
-e SKIP=MirrorPod \ # Run all conformance test and skip MirrorPod test
gcr.io/google_containers/node-test-amd64:v0.1
```

### Caveats

* The test will leave some docker images on the node, including the node
conformance test image and images of containers used in the functionality
test.
* The test will leave dead containers on the node, these containers are created
during the functionality test.
* Node conformance test validates kernel configuration. However, in some os
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be listed in the requirements?

distro the kernel module `configs` may not be loaded by default, and you will get
the error `no config path in [POSSIBLE KERNEL CONFIG FILE PATHS] is
available`. In that case please do either of the followings:
* Manually load/unload `configs` kernel module: run `sudo modprobe configs` to
load the kernel module, and `sudo modprobe -r configs` to unload it after the test.
* Mount `modprobe` into the container: Add option `-v /bin/kmod:/bin/kmod
-v /sbin/modprobe:/sbin/modprobe -v /lib/modules:/lib/modules` when starting
the test container.
2 changes: 1 addition & 1 deletion docs/admin/node-problem.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ it to [support other log format](/docs/admin/node-problem/#support-other-log-for

## Enable/Disable in GCE cluster

Node problem detector is [running as a cluster addon](cluster-large.md/#Addon-Resources) enabled by default in the
Node problem detector is [running as a cluster addon](cluster-large.md/#addon-resources) enabled by default in the
gce cluster.

You can enable/disable it by setting the environment variable
Expand Down