Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

toml plugin in 0.9.6 eats the first character of each key #3896

Closed
haraldg opened this issue Jun 19, 2021 · 9 comments · Fixed by #3901
Closed

toml plugin in 0.9.6 eats the first character of each key #3896

haraldg opened this issue Jun 19, 2021 · 9 comments · Fixed by #3901
Assignees
Labels
Milestone

Comments

@haraldg
Copy link

haraldg commented Jun 19, 2021

In updating the openwrt package to 0.9.6 I noticed that the ini-plugin has been removed and toml is recommended instead. However it seems each time the toml plugin writes to the file it eats the first character of each key:

Steps to Reproduce the Problem

root@OpenWrt:~/.config# kdb ls user:/
root@OpenWrt:~/.config# ls
root@OpenWrt:~/.config# kdb set user:/test/abcdef 1
Create a new key user:/test/abcdef with string "1"
root@OpenWrt:~/.config# ls
default.ecf
root@OpenWrt:~/.config# cat default.ecf 
est.abcdef = 1
root@OpenWrt:~/.config# kdb ls user:/
user:/est/abcdef
root@OpenWrt:~/.config# kdb set user:/test/xyz 2
Create a new key user:/test/xyz with string "2"
root@OpenWrt:~/.config# kdb ls user:/
user:/est/xyz
user:/st/abcdef
root@OpenWrt:~/.config# cat default.ecf 
st.abcdef = 1
est.xyz = 2

If your key database (KDB) might influence the outcome, please use kdb stash
to temporarily have an empty KDB. (Restore instructions are printed.)

Expected Result

The keys should remain stable across any calls to kdb set

Actual Result

Every time I use kdb set all keys got shorter by one character.

System Information

  • Elektra Version: 0.9.6
  • OpenWRT 21.02.0-rc3
@markus2330 markus2330 added the bug label Jun 20, 2021
@markus2330 markus2330 added this to the 0.9.7 milestone Jun 20, 2021
@markus2330
Copy link
Contributor

Thank you so much for reporting!

I can reproduce the bug. The problem seems to be specific to the mountpoint /, so this triggers the error:

kdb mount default.toml / toml
kdb set user:/test/abcdef 1
#> Create a new key user:/test/abcdef with string "1"
cat `kdb file user:/test`
est.abcdef = 1
kdb get user:/test                                                                                                                                                                                          
#> Did not find key 'user:/test'
kdb get user:/est/abcdef                                                                                                                                                                                  
#> 1
kdb rm user:/est/abcdef
kdb umount /

but with another mountpoint the problem does not occur:

kdb mount test.toml /test/toml toml 
kdb set user:/test/toml/test/abcdef 1
#> Create a new key user:/test/toml/test/abcdef with string "1"
cat `kdb file user:/test/toml/test/abcdef`                                                                                                                                                         
#> test.abcdef = 1
kdb rm user:/test/toml/test/abcdef                                                                                                                                                                         kdb umount /test/tom

My guess is that it is either a problem introduced in #3555 (keyname overhaul by @kodebach) or this case was simply never tested as toml is not yet being tested as default backend (#2330 @bauhaus93). Can you both take a look please?

@haraldg
Copy link
Author

haraldg commented Jun 20, 2021

Thanks for your confirmation and details.

I'm happy to use any other storage plugin as default for openwrt, but ini has already been removed and toml was recommended. The criteria for a default backend are:

  1. no dependency besides libc
  2. human readable file format

@kodebach
Copy link
Member

This certainly could be an error caused by #3555. Other plugins (e.g. dump) had similar issues with cascading keys. I don't have time do any real work on this right now, but I will check again later this week.

@markus2330 are you sure it is just caused by / or does it happen with any root key (e.g. user:/)? The TOML plugin seems to mostly use unescaped names, where cascading vs. non-cascading makes no difference. However, root keys are unique in that they contain an empty unescaped part at the end.

The criteria for a default backend are:

  1. no dependency besides libc
  2. human readable file format

You could try dump. It is technically human readable as long as all key values are human readable, but I wouldn't call it human-friendly or human-editable. It is however the most basic plugin that supports all of Elektra's features and is very well tested (it is used as the default plugin, with the default compile options).

@haraldg
Copy link
Author

haraldg commented Jun 20, 2021

When writing the initial package for OpenWRT we decided against dump as default storage on purpose: Aside from the human-friendlyness it actually has a dependency besides libc - libstdcpp.

@kodebach
Copy link
Member

it actually has a dependency besides libc - libstdcpp.

Ah yes, I forgot about that. The C++ dependency is of course not ideal for embedded systems.

kodebach added a commit to kodebach/libelektra that referenced this issue Jun 27, 2021
kodebach added a commit to kodebach/libelektra that referenced this issue Jun 27, 2021
@kodebach
Copy link
Member

I'm now 99% certain this issue is caused only by root-level keys. We should probably add a version of check_kdb_internal_suite that uses a root level key, since they are slightly different since the key name change. In particular, root level keys are the only ones that end with a / in escaped form, which translates to an additional \0 at the end of the unescaped form.

Maybe we should change the (un)escaping again so that user:/ is \x06\x00 instead of \x06\x00\x00. It would make certain sections of code (most importantly keyUnescapedName (key) + keyGetUnescapedNameSize (parent) for relative names) more straightforward. Not sure, if that is worth it though.

@markus2330
Copy link
Contributor

I'm now 99% certain this issue is caused only by root-level keys.

#3478 would give certainty.

We should probably add a version of check_kdb_internal_suite that uses a root level key

Yes, I agree. Very possible that other storage plugins face the same issue. Can you create an issue or PR for this?

Maybe we should change the (un)escaping again so that user:/ is \x06\x00 instead of \x06\x00\x00. It would make certain sections of code (most importantly keyUnescapedName (key) + keyGetUnescapedNameSize (parent) for relative names) more straightforward. Not sure, if that is worth it though.

Yes, very good idea. I also don't like the special handling of / keys. Can you create an issue or PR for this?

@markus2330 markus2330 reopened this Jun 28, 2021
@kodebach
Copy link
Member

See #3902, #3903

@kodebach
Copy link
Member

I'm closing this, since the original issue is fixed. I created #3910 for any future issues that prevent TOML from being used as the default storage plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants