bug in host-to-vector-quad on lispworks (also host-to-vector-quad)
Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that (get-hosts-by-name "localhost") returns the following list: (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1)) Then get-host-by-name simply takes the first of that list and returns it. Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) The error message is due to falling out of the ecase: #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO. I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost"). For example, I just ran this in the REPL -- log: USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r) (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)) This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it. However, I can say I do not notice other networking issues on my system. Thanks, Mark
Hi Mark, thanks for reporting this issue. It’s obvious that #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) means IPv6 address “::1”, which is indeed a valid address mapping to “localhost”. The function HOST-TO-HBO is not an exported function, it’s used internally by some backends and deals only with integers representing IPv4 addresses. But on LispWorks (>=6.1) the function lw-hbo-to-vector-quad may pass longer vectors back to HOST-TO-HBO and then caused this issue. On my Mac OS X laptop, I also got the same: (the first returned address is “::1”) CL-USER 5 > (comm:get-host-entry "localhost" :fields '(:addresses)) (#<COMM:IPV6-ADDRESS ::1 4020187E73> 2130706433) So what’s your desired behavior here? In the quick fixes, I could do something to make get-random-host-by-name return a random IPv4 address in presence of mixed IPv4/v6 address lists. But generally speaking get-hosts-by-name should support IPv6 addresses, especially for those IPv6-only hostnames. On the other side, HOST-TO-HBO is not an exported function, and it’s also not called by LispWorks backends, I don’t understand how your call chain finally touched it and let LispWorks throw an exception. It’s hard to support IPv6 in HOST-TO-HBO. Let me know what you think here, and with your help/inputs we can make usocket better. Regards, Chun
Il giorno 04 lug 2018, alle ore 21:04, Mark H. David <mhd@yv.org> ha scritto:
Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that
(get-hosts-by-name "localhost")
returns the following list:
(#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1))
Then get-host-by-name simply takes the first of that list and returns it.
Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)
The error message is due to falling out of the ecase:
#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO.
I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost"). For example, I just ran this in the REPL -- log:
USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r) (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))
This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it. However, I can say I do not notice other networking issues on my system.
Thanks, Mark
Hi, Kind of asking for a friend. I thought the thing to do would be to loop through the list and choose the first non-ipv6 result whereas now it simply chooses the first result. For the random host case, it should similarly filter out ipv6 entries in the result. Thank you very much! -Mark ----- Original message ----- From: "Chun Tian (binghe)" <binghe.lisp@gmail.com> To: "Mark H. David" <mhd@yv.org> Cc: "usocket-devel" <usocket-devel@common-lisp.net> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad) Date: Wed, 4 Jul 2018 22:47:04 +0200 Hi Mark, thanks for reporting this issue. It’s obvious that #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) means IPv6 address “::1”, which is indeed a valid address mapping to “localhost”. The function HOST-TO-HBO is not an exported function, it’s used internally by some backends and deals only with integers representing IPv4 addresses. But on LispWorks (>=6.1) the function lw-hbo-to-vector-quad may pass longer vectors back to HOST-TO-HBO and then caused this issue. On my Mac OS X laptop, I also got the same: (the first returned address is “::1”) CL-USER 5 > (comm:get-host-entry "localhost" :fields '(:addresses)) (#<COMM:IPV6-ADDRESS ::1 4020187E73> 2130706433) So what’s your desired behavior here? In the quick fixes, I could do something to make get-random-host-by-name return a random IPv4 address in presence of mixed IPv4/v6 address lists. But generally speaking get-hosts-by-name should support IPv6 addresses, especially for those IPv6-only hostnames. On the other side, HOST-TO-HBO is not an exported function, and it’s also not called by LispWorks backends, I don’t understand how your call chain finally touched it and let LispWorks throw an exception. It’s hard to support IPv6 in HOST-TO-HBO. Let me know what you think here, and with your help/inputs we can make usocket better. Regards, Chun
Il giorno 04 lug 2018, alle ore 21:04, Mark H. David <mhd@yv.org> ha scritto:
Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that
(get-hosts-by-name "localhost")
returns the following list:
(#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1))
Then get-host-by-name simply takes the first of that list and returns it.
Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)
The error message is due to falling out of the ecase:
#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO.
I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost"). For example, I just ran this in the REPL -- log:
USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r) (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))
This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it. However, I can say I do not notice other networking issues on my system.
Thanks, Mark
Email had 1 attachment: + signature.asc 1k (application/pgp-signature)
Hi Mark, Now I see your points. So IPv4 addresses should have a higher priority over IPv6 when returned by USOCKET host resolving functions. I’m a little busy in this week, *next* week I’ll revise all related USOCKET functions and make the changes (also need a better way to detect IPv6 addresses in various forms). Hopefully at the end of July I can make a new release with cumulative updates in the past... more than one year. —Chun
Il giorno 04 lug 2018, alle ore 23:44, Mark H. David <mhd@yv.org> ha scritto:
Hi, Kind of asking for a friend. I thought the thing to do would be to loop through the list and choose the first non-ipv6 result whereas now it simply chooses the first result. For the random host case, it should similarly filter out ipv6 entries in the result. Thank you very much! -Mark
----- Original message ----- From: "Chun Tian (binghe)" <binghe.lisp@gmail.com> To: "Mark H. David" <mhd@yv.org> Cc: "usocket-devel" <usocket-devel@common-lisp.net> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad) Date: Wed, 4 Jul 2018 22:47:04 +0200
Hi Mark,
thanks for reporting this issue.
It’s obvious that #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) means IPv6 address “::1”, which is indeed a valid address mapping to “localhost”. The function HOST-TO-HBO is not an exported function, it’s used internally by some backends and deals only with integers representing IPv4 addresses. But on LispWorks (>=6.1) the function lw-hbo-to-vector-quad may pass longer vectors back to HOST-TO-HBO and then caused this issue.
On my Mac OS X laptop, I also got the same: (the first returned address is “::1”)
CL-USER 5 > (comm:get-host-entry "localhost" :fields '(:addresses)) (#<COMM:IPV6-ADDRESS ::1 4020187E73> 2130706433)
So what’s your desired behavior here? In the quick fixes, I could do something to make get-random-host-by-name return a random IPv4 address in presence of mixed IPv4/v6 address lists. But generally speaking get-hosts-by-name should support IPv6 addresses, especially for those IPv6-only hostnames.
On the other side, HOST-TO-HBO is not an exported function, and it’s also not called by LispWorks backends, I don’t understand how your call chain finally touched it and let LispWorks throw an exception. It’s hard to support IPv6 in HOST-TO-HBO.
Let me know what you think here, and with your help/inputs we can make usocket better.
Regards,
Chun
Il giorno 04 lug 2018, alle ore 21:04, Mark H. David <mhd@yv.org> ha scritto:
Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that
(get-hosts-by-name "localhost")
returns the following list:
(#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1))
Then get-host-by-name simply takes the first of that list and returns it.
Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)
The error message is due to falling out of the ecase:
#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO.
I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost"). For example, I just ran this in the REPL -- log:
USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r) (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))
This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it. However, I can say I do not notice other networking issues on my system.
Thanks, Mark
Email had 1 attachment: + signature.asc 1k (application/pgp-signature)
Hi, that sounds very good to me. Thanks a lot. -Mark ----- Original message ----- From: "Chun Tian (binghe)" <binghe.lisp@gmail.com> To: "Mark H. David" <mhd@yv.org> Cc: usocket development <usocket-devel@common-lisp.net> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad) Date: Thu, 5 Jul 2018 14:18:54 +0200 Hi Mark, Now I see your points. So IPv4 addresses should have a higher priority over IPv6 when returned by USOCKET host resolving functions. I’m a little busy in this week, *next* week I’ll revise all related USOCKET functions and make the changes (also need a better way to detect IPv6 addresses in various forms). Hopefully at the end of July I can make a new release with cumulative updates in the past... more than one year. —Chun
Il giorno 04 lug 2018, alle ore 23:44, Mark H. David <mhd@yv.org> ha scritto:
Hi, Kind of asking for a friend. I thought the thing to do would be to loop through the list and choose the first non-ipv6 result whereas now it simply chooses the first result. For the random host case, it should similarly filter out ipv6 entries in the result. Thank you very much! -Mark
----- Original message ----- From: "Chun Tian (binghe)" <binghe.lisp@gmail.com> To: "Mark H. David" <mhd@yv.org> Cc: "usocket-devel" <usocket-devel@common-lisp.net> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad) Date: Wed, 4 Jul 2018 22:47:04 +0200
Hi Mark,
thanks for reporting this issue.
It’s obvious that #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) means IPv6 address “::1”, which is indeed a valid address mapping to “localhost”. The function HOST-TO-HBO is not an exported function, it’s used internally by some backends and deals only with integers representing IPv4 addresses. But on LispWorks (>=6.1) the function lw-hbo-to-vector-quad may pass longer vectors back to HOST-TO-HBO and then caused this issue.
On my Mac OS X laptop, I also got the same: (the first returned address is “::1”)
CL-USER 5 > (comm:get-host-entry "localhost" :fields '(:addresses)) (#<COMM:IPV6-ADDRESS ::1 4020187E73> 2130706433)
So what’s your desired behavior here? In the quick fixes, I could do something to make get-random-host-by-name return a random IPv4 address in presence of mixed IPv4/v6 address lists. But generally speaking get-hosts-by-name should support IPv6 addresses, especially for those IPv6-only hostnames.
On the other side, HOST-TO-HBO is not an exported function, and it’s also not called by LispWorks backends, I don’t understand how your call chain finally touched it and let LispWorks throw an exception. It’s hard to support IPv6 in HOST-TO-HBO.
Let me know what you think here, and with your help/inputs we can make usocket better.
Regards,
Chun
Il giorno 04 lug 2018, alle ore 21:04, Mark H. David <mhd@yv.org> ha scritto:
Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that
(get-hosts-by-name "localhost")
returns the following list:
(#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1))
Then get-host-by-name simply takes the first of that list and returns it.
Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)
The error message is due to falling out of the ecase:
#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO.
I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost"). For example, I just ran this in the REPL -- log:
USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r) (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))
This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it. However, I can say I do not notice other networking issues on my system.
Thanks, Mark
Email had 1 attachment: + signature.asc 1k (application/pgp-signature)
Email had 1 attachment: + signature.asc 1k (application/pgp-signature)
Hi Mark, I’ve made the changes in GET-HOST-BY-NAME and GET-RANDOM-HOST-BY-NAME on usocket trunk (master) [1]. Now they only returns IPv4 addresses if the underlying hostname resolves to both IPv4 and IPv6 addresses. I think this change is safe for all users. I also exported these functions as I saw many users directly use them as exported symbols. This is reasonable too, as GET-RANDOM-HOST-BY-NAME actually gives the ability to do load balancing. Any way, could you please confirm if the current code base works for you? —Chun [1] https://github.com/usocket/usocket <https://github.com/usocket/usocket>, a1f32274686d3749ce491dd768c9d0da1a4479c4
Il giorno 05 lug 2018, alle ore 20:13, Mark H. David <mhd@yv.org> ha scritto:
Hi, that sounds very good to me. Thanks a lot. -Mark
----- Original message ----- From: "Chun Tian (binghe)" <binghe.lisp@gmail.com> To: "Mark H. David" <mhd@yv.org> Cc: usocket development <usocket-devel@common-lisp.net> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad) Date: Thu, 5 Jul 2018 14:18:54 +0200
Hi Mark,
Now I see your points. So IPv4 addresses should have a higher priority over IPv6 when returned by USOCKET host resolving functions. I’m a little busy in this week, *next* week I’ll revise all related USOCKET functions and make the changes (also need a better way to detect IPv6 addresses in various forms). Hopefully at the end of July I can make a new release with cumulative updates in the past... more than one year.
—Chun
Il giorno 04 lug 2018, alle ore 23:44, Mark H. David <mhd@yv.org> ha scritto:
Hi, Kind of asking for a friend. I thought the thing to do would be to loop through the list and choose the first non-ipv6 result whereas now it simply chooses the first result. For the random host case, it should similarly filter out ipv6 entries in the result. Thank you very much! -Mark
----- Original message ----- From: "Chun Tian (binghe)" <binghe.lisp@gmail.com> To: "Mark H. David" <mhd@yv.org> Cc: "usocket-devel" <usocket-devel@common-lisp.net> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad) Date: Wed, 4 Jul 2018 22:47:04 +0200
Hi Mark,
thanks for reporting this issue.
It’s obvious that #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) means IPv6 address “::1”, which is indeed a valid address mapping to “localhost”. The function HOST-TO-HBO is not an exported function, it’s used internally by some backends and deals only with integers representing IPv4 addresses. But on LispWorks (>=6.1) the function lw-hbo-to-vector-quad may pass longer vectors back to HOST-TO-HBO and then caused this issue.
On my Mac OS X laptop, I also got the same: (the first returned address is “::1”)
CL-USER 5 > (comm:get-host-entry "localhost" :fields '(:addresses)) (#<COMM:IPV6-ADDRESS ::1 4020187E73> 2130706433)
So what’s your desired behavior here? In the quick fixes, I could do something to make get-random-host-by-name return a random IPv4 address in presence of mixed IPv4/v6 address lists. But generally speaking get-hosts-by-name should support IPv6 addresses, especially for those IPv6-only hostnames.
On the other side, HOST-TO-HBO is not an exported function, and it’s also not called by LispWorks backends, I don’t understand how your call chain finally touched it and let LispWorks throw an exception. It’s hard to support IPv6 in HOST-TO-HBO.
Let me know what you think here, and with your help/inputs we can make usocket better.
Regards,
Chun
Il giorno 04 lug 2018, alle ore 21:04, Mark H. David <mhd@yv.org> ha scritto:
Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that
(get-hosts-by-name "localhost")
returns the following list:
(#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1))
Then get-host-by-name simply takes the first of that list and returns it.
Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)
The error message is due to falling out of the ecase:
#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO.
I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost"). For example, I just ran this in the REPL -- log:
USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r) (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))
This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it. However, I can say I do not notice other networking issues on my system.
Thanks, Mark
Email had 1 attachment: + signature.asc 1k (application/pgp-signature)
Email had 1 attachment: + signature.asc 1k (application/pgp-signature)
participants (2)
-
Chun Tian (binghe) -
Mark H. David