feat: add torch dequantization for IQ1_S, IQ1_M, IQ2_XXS, IQ2_S, IQ3_XXS, IQ3_S

[octo-patch]

Closes #405

Problem

Lower IQ quant types (IQ1_S, IQ1_M, IQ2_XXS, IQ2_S, IQ3_XXS, IQ3_S) currently fall back to the slow numpy path in dequantize_tensor:

# this is incredibly slow
tqdm.write(f"Falling back to numpy dequant for qtype: ...")
new = gguf.quants.dequantize(tensor.cpu().numpy(), qtype)

This makes loading models quantized with these formats (e.g. Unsloth UD quants used as text encoders) very slow.

Solution

Implements native PyTorch dequantization functions for all six missing IQ quant types, using the quantization grids already provided by the gguf Python package.

The implementations follow the same pattern as the existing IQ4_NL and IQ4_XS dequantizers and were cross-referenced against the numpy reference implementation in gguf.quants.

Testing

All new functions are verified against gguf.quants.dequantize() (the numpy reference):

Q8_0:    max_diff=0.000000 ✓
Q6_K:    max_diff=0.000000 ✓
IQ4_NL:  max_diff=0.000000 ✓
IQ4_XS:  max_diff=0.000000 ✓
IQ3_S:   max_diff=0.000000 ✓
IQ3_XXS: max_diff=0.000000 ✓
IQ2_S:   max_diff=0.000000 ✓
IQ2_XXS: max_diff=0.000000 ✓
IQ1_M:   max_diff=0.000000 ✓
IQ1_S:   max_diff=0.000000 ✓

[SSJGabraham]

Tested this and it works well for IQ3_S in my setup.

[octo-patch]

Thanks for testing @SSJGabraham! Great to hear IQ3_S is working well. If you get a chance to try any of the other quant types (IQ1_S, IQ1_M, IQ2_XXS, IQ2_S, IQ3_XXS), would love to hear how those work for you too.

[SSJGabraham]

@octo-patch One thing I noticed is that when I first ran a test, I thought I was using a IQ3_S, but it was actually IQ3_M, which if I am not mistaken was not listed in your new methods or in your summary above, but it still worked without reverting to the slow numpy path. I didn't notice a slowdown and I did not get a stream of messages in my console about it. When I switched to the proper IQ3_S model I intended to use, performance was identical.

Was this expected?