Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto: stop gpg-agents after unit test #2008

Closed

Conversation

petermax2
Copy link
Member

@petermax2 petermax2 commented May 17, 2018

Purpose

This PR affects unit tests of crypto and fcrypt.

The general idea is to:

  1. set GNUPGHOME to a temporary directory,
  2. delete the temporary directory and all of its contents as last step of the unit tests, and
  3. shut down gpg-agent when the unit tests are done.

Step 2 instructs gpg-agent to shut down. A big thank you to @ingwinlu for finding out how to properly shut down the gpg-agent.

Closes #1928 .

Checklist

Check relevant points but please do not remove entries.
For docu fixes, spell checking, and similar nothing
needs to be checked.

  • commit messages are fine ("module: short statement" syntax and refer to issues)
  • I added unit tests
  • I ran all tests locally and everything went fine
  • affected documentation is fixed
  • I added code comments, logging, and assertions (see doc/CODING.md)
  • meta data is updated (e.g. README.md of plugins)
  • release notes are updated (doc/news/_preparation_next_release.md)

TODOs

  • improve code quality
  • adapt fcrypt unit test


static void clean_env (void)
{
// TODO make it pretty
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check if gpg_dir != NULL

succeed_if (gpg_dir != NULL, "no memory available");
if (gpg_dir == NULL) exit (-1);
snprintf (gpg_dir, gpg_dir_size, "%s", tmp_template);
gpg_dir = mkdtemp (gpg_dir);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate code: line 126

@petermax2
Copy link
Member Author

jenkins build all please

@petermax2
Copy link
Member Author

Now I run consistently into the same problem as in #1973 . For some reason gpg returns with exit code 512 but the other tests suceed indicating, that the import was successful nevertheless.

@ingwinlu
Copy link
Contributor

ingwinlu commented May 20, 2018

I have a few ideas that we could still try:

  • Did you manage to get any useful debugging output?
  • Try another key?
  • start gpg-agent explicitly maybe that gives any logging output
  • if stderr does not give output, maybe add --logger-fd to write to a file
  • time problems? I am not sure the time is properly set time is set to host OS without TZ
  • try to import the key directly via jenkinsfile command, maybe that produces something loggable

no output on stderr really does not help...

@ingwinlu
Copy link
Contributor

ingwinlu commented May 23, 2018

another thing i noticed while trying through steps that might fail is that gpg will throw a warning regarding directory permissions when using GNUPGHOME on a random directory. But no error code is set.

//EDIT: and I only tried with the test_key.asc directly, not the hexdump in test_key.h
//EDIT2: tried manually setting GNUPGHOME and it worked as expected. still not closer to why gpg returns that error code on the test server.

@markus2330
Copy link
Contributor

So still no progress here?

@ingwinlu
Copy link
Contributor

ingwinlu commented May 23, 2018 via email

@petermax2
Copy link
Member Author

petermax2 commented May 23, 2018

So still no progress here?

I too have no progress so far on this issue. 😞

@ingwinlu
Copy link
Contributor

I just had something fun happen in https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/PR-2003/97/pipeline/326.

gpg tests randomly failed. replayed the test and it passed.

maybe trying to solve the gpg-agent issue is just exposing something wonky in the gpg interaction?

@petermax2
Copy link
Member Author

maybe trying to solve the gpg-agent issue is just exposing something wonky in the gpg interaction?

I will play around with the gpg module to find out more.

@ingwinlu
Copy link
Contributor

I know it would be a lot of work but have you considered replacing gpg exe manipulation with https://www.gnupg.org/software/gpgme/index.html?

@petermax2
Copy link
Member Author

Yes, I already had a discussion with @markus2330 . Personally I would prefer to use gpgme. But it would take a lot of work to replace the current gpg module.

@markus2330
Copy link
Contributor

Maybe writing a "hello world" testcase with gpgme would be a good start. Then we see if gpgme actually solves this problem. gpgme seems to work very similar to @petermax2's code.

@petermax2
Copy link
Member Author

gpgme seems to work very similar to @petermax2's code.

gpgme uses a lot more abstraction layers than my code 😆 LibAssuan handles the inter process communication but I did not have an in-depth look into its code so far.

I will try to advance my progress on the gpgme plugin prototype. Then we see if we encounter similar quirks on the build server.

@ingwinlu
Copy link
Contributor

@petermax2 I am not sure you followed #2056 but as you can see there the files for gpg-agent have changed and thus are not getting cleaned up anymore.

Changing the gpg module to cleanup itself is probably overkill as the c testframework should handle it and is just outdated.

Additionally I am sure you might have noticed that ftw is not 100% what you want here as it does return directories before the nested files (and directories). Hence the rmdir call will not be executed.
This will probably stop gpg-agents because their socket files are removed, but the directories are not cleaned up completely.

@petermax2
Copy link
Member Author

petermax2 commented Jun 13, 2018

I am not sure you followed #2056 but as you can see there the files for gpg-agent have changed and thus are not getting cleaned up anymore.

I did.

Changing the gpg module to cleanup itself is probably overkill

It is also not acceptable for a regular user session. I do not want my gpg-agent to stop after using Elektra. Especially if my agent was running before, I would be surprised if it is closed after using Elektra.

Additionally I am sure you might have noticed that

Actually I did some sort of dirty hack by invoking it three times. The first one deletes all files, the scond one the sub directories and the third one the root directory.

+static void clean_env (void)
+{
+	if (gpg_dir)
+	{
+		// 1. delete files
+		ftw (gpg_dir, delete_dir_content, 50);
+		// 2. delete sub-directories
+		ftw (gpg_dir, delete_dir_content, 50);
+		// 3. delete root directory (GNUPGHOME)
+		ftw (gpg_dir, delete_dir_content, 50);
+		free (gpg_dir);
+	}

EDIT: I do appreciate every input about how to make this code less hacky.

petermax2 added a commit to petermax2/libelektra that referenced this pull request Jun 30, 2018
petermax2 added a commit to petermax2/libelektra that referenced this pull request Jul 15, 2018
sanssecours pushed a commit to sanssecours/elektra that referenced this pull request Aug 20, 2018
petermax2 added a commit to petermax2/libelektra that referenced this pull request Aug 21, 2018
@petermax2 petermax2 deleted the crypto_fix_agents branch October 7, 2018 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tests spawn unlimited gpg-agents
3 participants