No that wouldn't affect response times. The number of rules in a policy can have a minuscule impact on how long it takes to make a decision but we are talking in the nanosecond range. You'd need to have an absolutely massive policy file (100's of megabytes) in order to see any noticeable effect.