debugging dhcp for pihole

On my laptop, to request a new ip address from the dhcp server:

doas udhcpc -i wlan0 -f -v

on the rpi, I can look at incomming dhcp requests:

$ doas tcpdump -i enu1u1u1 port 67 or port 68
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enu1u1u1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
03:06:44.449485 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 08:71:90:17:eb:cf (oui Unknown), length 300
03:06:44.450699 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 08:71:90:17:eb:cf (oui Unknown), length 300
03:06:44.490692 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 08:71:90:17:eb:cf (oui Unknown), length 300
03:06:44.491317 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 08:71:90:17:eb:cf (oui Unknown), length 300

From this output, it looks like the rpi is getting the requests, but it is not responding with a reply. Pihole is running on the rpi in a podman container. The compose.yaml looks like:

$ cat compose.yaml
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    privileged: true
    network_mode: host
    # Remove all port mappings
    environment:
      TZ: 'US/Central'
      FTLCONF_webserver_api_password: 'correct horse battery staple'
      FTLCONF_dns_listeningMode: 'all'
      DHCP_ACTIVE: 'true'
      # Add your network settings
      DHCP_START: '192.168.1.65'
      DHCP_END: '192.168.1.253'
      DHCP_ROUTER: '192.168.1.1'
    volumes:
      - './etc-pihole:/etc/pihole'
    cap_add:
      - NET_ADMIN
      - SYS_TIME
      - SYS_NICE
    restart: unless-stopped

So my next thought is to try the tcpdump program from within the pihole container:

$ podman exec -it pihole bash
squiggle:/# pidof pihole-FTL # to verify that pihole-FTL is running
49
squiggle:/#

tcpdump is not installed within the pihole container so I'll skip it for now to run the builtin pihole diagnosis tool. Everything looked mostly normal, but I got a few interesting leads:

$ pihole -d
...
*** [ DIAGNOSING ]: Networking
[i] Default IPv4 gateway(s):
     192.168.1.254%enu1u1u1
   * Pinging first gateway 192.168.1.254...
ping: permission denied (are you root?)
[✗] Gateway did not respond. (https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546)

[i] Default IPv6 gateway(s):
     fe80::6e63:9cff:fecf:b830%enu1u1u1
   * Pinging first gateway fe80::6e63:9cff:fecf:b830%enu1u1u1...
ping6: permission denied (are you root?)
[✗] Gateway did not respond. (https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546)
...

...
   [dhcp]
     active = true ### CHANGED, default = false
     start = "192.168.1.65" ### CHANGED, default = ""
     end = "192.168.1.253" ### CHANGED, default = ""
     router = "192.168.1.1" ### CHANGED, default = ""
     netmask = ""
     leaseTime = ""
...

From this, I can see that dhcp server is supposed to be active, and we are getting some errors reaching the default gateways. 192.168.1.254 is the router ip and should be correct, but we are getting a permission denied on the ping. The link only explains the importence of why we need to set the gateway properly. I don't see any information online about others getting this same ping error in their diagnostics. If I'm getting a permissions issue here, is it possible I'm getting the same permissions issue somehwere else, too?

Interesting, so if I tail the pihole log I get this (with unecessary stuff cut):

$ tail -F /var/log/pihole/pihole.log
2025-03-18 03:02:03.010 config 192.168.1.221 is NXDOMAIN
2025-03-18 03:02:37.640 DHCPDISCOVER(enu1u1u1) 22:f9:ac:4a:9b:96
2025-03-18 03:02:37.640 DHCPOFFER(enu1u1u1) 192.168.1.94 22:f9:ac:4a:9b:96
2025-03-18 03:02:37.640 ARP-cache injection failed: Operation not permitted
2025-03-18 03:02:37.641 DHCPDISCOVER(enu1u1u1) 22:f9:ac:4a:9b:96
2025-03-18 03:02:37.641 DHCPOFFER(enu1u1u1) 192.168.1.94 22:f9:ac:4a:9b:96
2025-03-18 03:02:37.641 ARP-cache injection failed: Operation not permitted
2025-03-18 03:02:38.661 DHCPDISCOVER(enu1u1u1) 22:f9:ac:4a:9b:96
2025-03-18 03:02:38.661 DHCPOFFER(enu1u1u1) 192.168.1.94 22:f9:ac:4a:9b:96
2025-03-18 03:02:38.661 ARP-cache injection failed: Operation not permitted
2025-03-18 03:02:38.662 DHCPDISCOVER(enu1u1u1) 22:f9:ac:4a:9b:96
2025-03-18 03:02:38.662 DHCPOFFER(enu1u1u1) 192.168.1.94 22:f9:ac:4a:9b:96
2025-03-18 03:02:38.662 ARP-cache injection failed: Operation not permitted
2025-03-18 03:02:38.688 DHCPREQUEST(enu1u1u1) 192.168.1.241 22:f9:ac:4a:9b:96
2025-03-18 03:02:38.689 DHCPNAK(enu1u1u1) 192.168.1.241 22:f9:ac:4a:9b:96 wrong server-ID
2025-03-18 03:02:38.690 DHCPREQUEST(enu1u1u1) 192.168.1.241 22:f9:ac:4a:9b:96
2025-03-18 03:02:38.690 DHCPNAK(enu1u1u1) 192.168.1.241 22:f9:ac:4a:9b:96 wrong server-ID

The thing that looks interesting to me is the ARP-cache injection failed: Operation not permitted

This error, I did find online; the solution being to simply run the pihole container as root, which did work for me. I'm not sure if this is the best option or not, but I'm happy it's finally working. I should have just checked the logs first!

doas podman-compose up -d