Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search repository tags using --list-tags #7836

Merged
merged 1 commit into from
Oct 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions cmd/podman/images/search.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ func searchFlags(flags *pflag.FlagSet) {
flags.BoolVar(&searchOptions.NoTrunc, "no-trunc", false, "Do not truncate the output")
flags.StringVar(&searchOptions.Authfile, "authfile", auth.GetDefaultAuthFile(), "Path of the authentication file. Use REGISTRY_AUTH_FILE environment variable to override")
flags.BoolVar(&searchOptions.TLSVerifyCLI, "tls-verify", true, "Require HTTPS and verify certificates when contacting registries")
flags.BoolVar(&searchOptions.ListTags, "list-tags", false, "List the tags of the input registry")
}

// imageSearch implements the command for searching images.
Expand All @@ -101,6 +102,10 @@ func imageSearch(cmd *cobra.Command, args []string) error {
return errors.Errorf("Limit %d is outside the range of [1, 100]", searchOptions.Limit)
}

if searchOptions.ListTags && len(searchOptions.Filters) != 0 {
return errors.Errorf("filters are not applicable to list tags result")
}

// TLS verification in c/image is controlled via a `types.OptionalBool`
// which allows for distinguishing among set-true, set-false, unspecified
// which is important to implement a sane way of dealing with defaults of
Expand All @@ -119,12 +124,19 @@ func imageSearch(cmd *cobra.Command, args []string) error {
if err != nil {
return err
}

if len(searchReport) == 0 {
return nil
}

hdrs := report.Headers(entities.ImageSearchReport{}, nil)
row := "{{.Index}}\t{{.Name}}\t{{.Description}}\t{{.Stars}}\t{{.Official}}\t{{.Automated}}\n"
if searchOptions.ListTags {
if len(searchOptions.Filters) != 0 {
return errors.Errorf("filters are not applicable to list tags result")
}
row = "{{.Name}}\t{{.Tag}}\n"
}
if cmd.Flags().Changed("format") {
row = report.NormalizeFormat(searchOptions.Format)
}
Expand Down
1 change: 1 addition & 0 deletions completions/bash/podman
Original file line number Diff line number Diff line change
Expand Up @@ -2024,6 +2024,7 @@ _podman_search() {
--help
-h
--no-trunc
--list-tags
"
_complete_ "$options_with_args" "$boolean_options"
}
Expand Down
18 changes: 18 additions & 0 deletions docs/source/markdown/podman-search.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ Valid placeholders for the Go template are listed below:
| .Stars | Star count of image |
| .Official | "[OK]" if image is official |
| .Automated | "[OK]" if image is automated |
| .Tag | Repository tag |

Note: use .Tag only if the --list-tags is set.

**--limit**=*limit*

Expand All @@ -65,6 +68,12 @@ Example if limit is 10 and two registries are being searched, the total
number of results will be 20, 10 from each (if there are at least 10 matches in each).
The order of the search results is the order in which the API endpoint returns the results.

**--list-tags**

List the available tags in the repository for the specified image.
**Note:** --list-tags requires the search term to be a fully specified image name.
The result contains the Image name and its tag, one line for every tag associated with the image.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all very contradictory. You're saying at the start that you'll list all tags for each image found in the repository, and then that the full name must be used. I think from our conversations that the image name with the repository must be supplied. I.e. you must do docker.io/library/alpine and you can't do docker.io/library/ or alpine. Given that assumption:

Suggested change
List the available tags in the repository for the specified image. **Note:** --list-tags requires the search term to be a fully
specified image name. The result contains the Image name and its tag, one line for every tag associated with the image.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the docker.io/library/alpine refer to a repository name, if append a tag to it, it will refer to an image?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@QiWang19 yeah, it's confusing and I may not have helped matters. When you push a container image to a registry, it creates a repository from it. How's this?

List the available tags in the repository for the specified image. **Note:** --list-tags requires the search term to be a fully
specified image. The result contains the Image name and its tag, one line for every tag found in the repository.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍
nicely done @QiWang19

**--no-trunc**

Do not truncate the output
Expand Down Expand Up @@ -140,6 +149,15 @@ fedoraproject.org registry.fedoraproject.org/f25/kubernetes-proxy
fedoraproject.org registry.fedoraproject.org/f25/kubernetes-scheduler 0
fedoraproject.org registry.fedoraproject.org/f25/mariadb 0
```

```
$ podman search --list-tags registry.redhat.io/rhel
NAME TAG
registry.redhat.io/rhel 7.3-74
registry.redhat.io/rhel 7.6-301
registry.redhat.io/rhel 7.1-9
...
```
TomSweeneyRedHat marked this conversation as resolved.
Show resolved Hide resolved
Note: This works only with registries that implement the v2 API. If tried with a v1 registry an error will be returned.

## FILES
Expand Down
51 changes: 51 additions & 0 deletions libpod/image/search.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ package image

import (
"context"
"fmt"
"strconv"
"strings"
"sync"

"github.com/containers/image/v5/docker"
"github.com/containers/image/v5/transports/alltransports"
"github.com/containers/image/v5/types"
sysreg "github.com/containers/podman/v2/pkg/registries"
"github.com/pkg/errors"
Expand Down Expand Up @@ -34,6 +36,8 @@ type SearchResult struct {
Official string
// Automated indicates if the image was created by an automated build.
Automated string
// Tag is the image tag
Tag string
}

// SearchOptions are used to control the behaviour of SearchImages.
Expand All @@ -49,6 +53,8 @@ type SearchOptions struct {
Authfile string
// InsecureSkipTLSVerify allows to skip TLS verification.
InsecureSkipTLSVerify types.OptionalBool
// ListTags returns the search result with available tags
ListTags bool
}

// SearchFilter allows filtering the results of SearchImages.
Expand Down Expand Up @@ -147,6 +153,15 @@ func searchImageInRegistry(term string, registry string, options SearchOptions)
// every types.SystemContext, and to compute the value just once in one
// place.
sc.SystemRegistriesConfPath = sysreg.SystemRegistriesConfPath()
if options.ListTags {
TomSweeneyRedHat marked this conversation as resolved.
Show resolved Hide resolved
results, err := searchRepositoryTags(registry, term, sc, options)
if err != nil {
logrus.Errorf("error listing registry tags %q: %v", registry, err)
return []SearchResult{}
}
return results
}

results, err := docker.SearchRegistry(context.TODO(), sc, registry, term, limit)
if err != nil {
logrus.Errorf("error searching registry %q: %v", registry, err)
Expand Down Expand Up @@ -207,6 +222,42 @@ func searchImageInRegistry(term string, registry string, options SearchOptions)
return paramsArr
}

func searchRepositoryTags(registry, term string, sc *types.SystemContext, options SearchOptions) ([]SearchResult, error) {
dockerPrefix := fmt.Sprintf("%s://", docker.Transport.Name())
imageRef, err := alltransports.ParseImageName(fmt.Sprintf("%s/%s", registry, term))
if err == nil && imageRef.Transport().Name() != docker.Transport.Name() {
return nil, errors.Errorf("reference %q must be a docker reference", term)
} else if err != nil {
imageRef, err = alltransports.ParseImageName(fmt.Sprintf("%s%s", dockerPrefix, fmt.Sprintf("%s/%s", registry, term)))
if err != nil {
return nil, errors.Errorf("reference %q must be a docker reference", term)
}
}
tags, err := docker.GetRepositoryTags(context.TODO(), sc, imageRef)
if err != nil {
return nil, errors.Errorf("error getting repository tags: %v", err)
}
limit := maxQueries
if len(tags) < limit {
limit = len(tags)
}
if options.Limit != 0 {
limit = len(tags)
if options.Limit < limit {
limit = options.Limit
}
}
paramsArr := []SearchResult{}
for i := 0; i < limit; i++ {
params := SearchResult{
Name: imageRef.DockerReference().Name(),
Tag: tags[i],
}
paramsArr = append(paramsArr, params)
}
return paramsArr, nil
}

// ParseSearchFilter turns the filter into a SearchFilter that can be used for
// searching images.
func ParseSearchFilter(filter []string) (*SearchFilter, error) {
Expand Down
7 changes: 5 additions & 2 deletions pkg/api/handlers/libpod/images.go
Original file line number Diff line number Diff line change
Expand Up @@ -608,6 +608,7 @@ func SearchImages(w http.ResponseWriter, r *http.Request) {
NoTrunc bool `json:"noTrunc"`
Filters []string `json:"filters"`
TLSVerify bool `json:"tlsVerify"`
ListTags bool `json:"listTags"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need to extend the swagger docs in pkg/api/server/register_images.go

}{
// This is where you can override the golang default value for one of fields
}
Expand All @@ -618,8 +619,9 @@ func SearchImages(w http.ResponseWriter, r *http.Request) {
}

options := image.SearchOptions{
Limit: query.Limit,
NoTrunc: query.NoTrunc,
Limit: query.Limit,
NoTrunc: query.NoTrunc,
ListTags: query.ListTags,
}
if _, found := r.URL.Query()["tlsVerify"]; found {
options.InsecureSkipTLSVerify = types.NewOptionalBool(!query.TLSVerify)
Expand Down Expand Up @@ -650,6 +652,7 @@ func SearchImages(w http.ResponseWriter, r *http.Request) {
reports[i].Stars = searchResults[i].Stars
reports[i].Official = searchResults[i].Official
reports[i].Automated = searchResults[i].Automated
reports[i].Tag = searchResults[i].Tag
}

utils.WriteResponse(w, http.StatusOK, reports)
Expand Down
4 changes: 4 additions & 0 deletions pkg/api/server/register_images.go
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,10 @@ func (s *APIServer) registerImagesHandlers(r *mux.Router) error {
// - `is-automated=(true|false)`
// - `is-official=(true|false)`
// - `stars=<number>` Matches images that has at least 'number' stars.
// - in: query
// name: listTags
// type: boolean
// description: list the available tags in the repository
// produces:
// - application/json
// responses:
Expand Down
1 change: 1 addition & 0 deletions pkg/bindings/images/images.go
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,7 @@ func Search(ctx context.Context, term string, opts entities.ImageSearchOptions)
params.Set("term", term)
params.Set("limit", strconv.Itoa(opts.Limit))
params.Set("noTrunc", strconv.FormatBool(opts.NoTrunc))
params.Set("listTags", strconv.FormatBool(opts.ListTags))
for _, f := range opts.Filters {
params.Set("filters", f)
}
Expand Down
4 changes: 4 additions & 0 deletions pkg/domain/entities/images.go
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,8 @@ type ImageSearchOptions struct {
NoTrunc bool
// SkipTLSVerify to skip HTTPS and certificate verification.
SkipTLSVerify types.OptionalBool
// ListTags search the available tags of the repository
ListTags bool
}

// ImageSearchReport is the response from searching images.
Expand All @@ -230,6 +232,8 @@ type ImageSearchReport struct {
Official string
// Automated indicates if the image was created by an automated build.
Automated string
// Tag is the repository tag
Tag string
}

// Image List Options
Expand Down
2 changes: 2 additions & 0 deletions pkg/domain/infra/abi/images.go
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,7 @@ func (ir *ImageEngine) Search(ctx context.Context, term string, opts entities.Im
Limit: opts.Limit,
NoTrunc: opts.NoTrunc,
InsecureSkipTLSVerify: opts.SkipTLSVerify,
ListTags: opts.ListTags,
}

searchResults, err := image.SearchImages(term, searchOpts)
Expand All @@ -529,6 +530,7 @@ func (ir *ImageEngine) Search(ctx context.Context, term string, opts entities.Im
reports[i].Stars = searchResults[i].Stars
reports[i].Official = searchResults[i].Official
reports[i].Automated = searchResults[i].Automated
reports[i].Tag = searchResults[i].Tag
}

return reports, nil
Expand Down
20 changes: 20 additions & 0 deletions test/e2e/search_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -423,4 +423,24 @@ registries = ['{{.Host}}:{{.Port}}']`
Expect(search.ExitCode()).To(Equal(0))
Expect(len(search.OutputToStringArray()) > 1).To(BeTrue())
})

It("podman search repository tags", func() {
search := podmanTest.Podman([]string{"search", "--list-tags", "--limit", "30", "docker.io/library/alpine"})
search.WaitWithDefaultTimeout()
Expect(search.ExitCode()).To(Equal(0))
Expect(len(search.OutputToStringArray())).To(Equal(31))

search = podmanTest.Podman([]string{"search", "--list-tags", "docker.io/library/alpine"})
search.WaitWithDefaultTimeout()
Expect(search.ExitCode()).To(Equal(0))
Expect(len(search.OutputToStringArray()) > 2).To(BeTrue())

search = podmanTest.Podman([]string{"search", "--filter=is-official", "--list-tags", "docker.io/library/alpine"})
search.WaitWithDefaultTimeout()
Expect(search.ExitCode()).To(Not(Equal(0)))

search = podmanTest.Podman([]string{"search", "--list-tags", "docker.io/library/"})
search.WaitWithDefaultTimeout()
Expect(len(search.OutputToStringArray()) == 0).To(BeTrue())
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to add a search for a registry too with a backslash only, no image name. For that and for this test, add a test to make sure the returned lines are greater than 1 and perhaps greater than 2 or 3.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you added the test with lines >2, ty! Could you also add a test that lists the tags for docker.io/library/ (no image, just a backslash). According to the doc, it's allowed: podman search registry.fedoraproject.org/ and if it is NOT allowed with the --list-tags option, it would be good to test that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test added. It's not allowed, no tags will be returned for docker.io/library/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with not allowing it, but we really should add a note to the --list-tags option in the man page saying that, and preferably raise some kind of error message telling the user that they need to specify an image or at least return an "image not found" kind of message.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hope I'm not being too painful @QiWang19 !

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the error would be "library/" must be a docker reference". Add to the doc "the search term should be repository name". Would this work?
$ bin/podman search --list-tags docker.io/library/
ERRO[0000] error listing registry tags "docker.io": reference "library/" must be a docker reference

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, first off, I'd still like to see a test for docker.io/library/ (or some other registry) and then test for the appropriate error message. I'm finding the man page to be very confusing at the moment, will try to suggest a rewrite.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

search does not return error, it logs errors to standard logger and returns the 0 entry of the result.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good and it looks like you've added a test for that, TY!

})