Change #263175

Comments

lib: make resolving HTTPS DNS records reliable:
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
  tried to get HTTPS-RRs is no answer for a resolve that wants
  it.
- fix late arrivals of resolve answers to match the "async"
  records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
  same time. We may need to resolve an IP interface while the
  main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
  available, even if A/AAAA queries are still ongoing.

For this, the "async" infrastructure is changed:

- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
  and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
  nothing about HTTPS.
  Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
  resolves inside one multi. This is weak, as the id may
  wrap around. However it is combined with the `mid` of
  the easy handle, making collisions highly unlikely.
  `data->state.async` is only accessed in few places where
  the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
  retrieve the HTTPS-RR information from the dns connection filter.
  Delay the connect if the HTTPS-RR is needed, but has not
  been resolved yet.

The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.

Details:

c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.

Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.

Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.

Closes #21175

Changed files

• docs/libcurl/curl_global_trace.md
• lib/asyn-ares.c
• lib/asyn-base.c
• lib/asyn-thrdd.c
• lib/asyn.h
• lib/cf-dns.c
• lib/cf-dns.h
• lib/cf-https-connect.c
• lib/cf-socket.c
• lib/connect.c
• lib/curl_trc.c
• lib/curl_trc.h
• lib/dnscache.c
• lib/dnscache.h
• lib/doh.c
• lib/doh.h
• lib/easy.c
• lib/ftp.c
• lib/hostip.c
• lib/hostip.h
• lib/hostip4.c
• lib/hostip6.c
• lib/httpsrr.c
• lib/httpsrr.h
• lib/multi.c
• lib/multihandle.h
• lib/request.c
• lib/setopt.c
• lib/socks.c
• lib/thrdpool.c
• lib/thrdpool.h
• lib/thrdqueue.c
• lib/thrdqueue.h
• lib/url.c
• lib/urldata.h
• lib/vtls/openssl.c
• lib/vtls/rustls.c
• lib/vtls/wolfssl.c
• tests/libtest/lib655.c
• tests/unit/unit1658.c