Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler container failed to find requires authentication #2433

Closed
jinlinGuan opened this issue Mar 12, 2020 · 9 comments 路 Fixed by #2441
Closed

Scheduler container failed to find requires authentication #2433

jinlinGuan opened this issue Mar 12, 2020 · 9 comments 路 Fixed by #2441
Assignees
Labels
3-high priority denoting release-blocking issues bug Something isn't working geneva support-services
Projects

Comments

@jinlinGuan
Copy link
Contributor

馃悶 Bug Report

Affected Services

The issue is located in: support-scheduler

Is this a regression?

-

Description and Minimal Reproduction

Refer to developer-scripts#233

馃敟 Exception or Error

level=ERROR ts=2020-03-11T08:08:16.1829089Z app=edgex-support-scheduler source=init.go:63 msg="Failed to load schedules and events command find requires authentication"

馃實 Your Environment

Deployment Environment:

EdgeX Version:
Master

Anything else relevant?
This issue is currently blocking mongo no secuity Jenkins jobs.
https://jenkins.edgexfoundry.org/view/blackbox-testing/job/blackbox-testing-centos7-blackbox-4c-2g-master/DATABASE=mongo,node=centos7-blackbox-4c-2g/6/console

@jinlinGuan jinlinGuan added the bug Something isn't working label Mar 12, 2020
@weichou1229
Copy link
Member

It seems didn't retrieve the username and password from the consul.

The steps to reproduce:

  1. Deploy EdgeX
docker-compose -f docker-compose-nexus-mongo-no-secty.yml up -d

https://github.com/edgexfoundry/developer-scripts/blob/master/releases/nightly-build/compose-files/docker-compose-nexus-mongo-no-secty.yml

  1. Navigate to http://localhost:48080/api/v1/reading
curl http://localhost:48080/api/v1/reading 
command find requires authentication
null
  1. I can find Username and password on the Consul
    http://localhost:8500/ui/dc1/kv/edgex/core/1.0/edgex-core-data/Databases/Primary/

  2. Navigate to http://localhost:48080/api/v1/config
    Username and password are both empty.

{
   "Service" : {
      ...
   },
   ...
   "Databases" : {
      "Primary" : {
         "Username" : "",
         "Password" : "",
         "Host" : "edgex-mongo",
         "Type" : "mongodb",
         "Name" : "coredata",
         "Timeout" : 501,
         "Port" : 27017
      }
   },
}

@bnevis-i
Copy link
Collaborator

bnevis-i commented Mar 17, 2020

Wondering out loud if this has something to do with what is present in types.go in go-mod-bootstrap. Note that the credentials are referenced via a nested struct instead of being included literally. This may be throwing off whatever kind of type introspection the code is doing to fill this struct up.

type Database struct {
	Credentials
	Type    string
	Timeout int
	Host    string
	Port    int
	Name    string
}

// Credentials encapsulates username-password attributes.
type Credentials struct {
	Username string
	Password string
}

@weichou1229
Copy link
Member

@bnevis-i
You are right, I change the Database struct

type Database struct {
	Username string
	Password string
	Type    string
	Timeout int
	Host    string
	Port    int
	Name    string
}

Then It can fetch the username password from Consul

	decoder := client.newConsulDecoder()
	decoder.Consul = client.consulConfig
        ...

	go decoder.Run()

	select {
	...
	case raw := <-updateChannel:
		configuration = raw
	}

	return configuration, err

https://github.com/edgexfoundry/go-mod-configuration/blob/master/internal/pkg/consul/client.go#L175

@bnevis-i
Copy link
Collaborator

bnevis-i commented Mar 18, 2020

@weichou1229 Great news! I tried to fork the module, but then had to update all the edgexfoundry/go-mod-bootstrap references to my fork everywhere. Been trying to get this going for hours. I will abandon my futile attempt to validate this given your result and let you push the necessary fixes.

@weichou1229
Copy link
Member

weichou1229 commented Mar 18, 2020

But I 'm not sure that the modification is correct.
@bnevis-i @tsconn23 Is it a correct way to fix the issue?

@bnevis-i
Copy link
Collaborator

When I send this the old data structure through fmt.Printf %v, the generic structure outputs as a nested data structure { { username, password } type, timeout, host, port name } whereas flattening it should remove the artificial nesting. Since before the mongo --auth fix, the database didn't require any password at all, I would guess that this implementation never worked, and that in the security-enabled case, passwords were injected via the secret store. This behavior seems consistent with the failures we are observing with blackbox-testing. I am not enough of a golang expert to figure out how to subvert go's module system to be able to test this change out easily at a system level. @tsconn23, do you think we can try this fix and roll it back if it doesn't improve the situation?

@michaelestrin
Copy link
Member

Based on what I've read, the proposed correction -- flattening the data structure -- is the correct one. Raise a PR. @weichou1229

@bnevis-i go.mod's replace should help you avoid making mass import name changes for testing in the future.

@weichou1229
Copy link
Member

PR opened edgexfoundry/go-mod-bootstrap#57

@bnevis-i
Copy link
Collaborator

bnevis-i commented Mar 18, 2020

   curl  http://localhost:48080/api/v1/config | python -mjson.tool

    "Databases": {
        "Primary": {
            "Host": "edgex-mongo",
            "Name": "coredata",
            "Password": "password",
            "Port": 27017,
            "Timeout": 5000,
            "Type": "mongodb",
            "Username": "core"
        }
    },

Configuration is now passing through after local testing with new go-mod-bootstrap. Continuing to test.

@michaelestrin michaelestrin added 3-high priority denoting release-blocking issues geneva support-services labels Mar 19, 2020
@michaelestrin michaelestrin added this to New Issues in Core WG via automation Mar 19, 2020
@michaelestrin michaelestrin added this to New Issues in Security WG via automation Mar 19, 2020
@michaelestrin michaelestrin moved this from New Issues to In progress in Core WG Mar 19, 2020
@michaelestrin michaelestrin moved this from New Issues to In progress in Security WG Mar 19, 2020
@AnthonyMBonafide AnthonyMBonafide moved this from In progress to QA/Code Review in Core WG Mar 20, 2020
Core WG automation moved this from QA/Code Review to Done Mar 23, 2020
Security WG automation moved this from In progress to Done Mar 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3-high priority denoting release-blocking issues bug Something isn't working geneva support-services
Projects
Security WG
  
Done
Development

Successfully merging a pull request may close this issue.

4 participants