yes, it matters. Most batteries specify a maximum charge current of 0.5C to 2C, with 1C being the most common. The C means the battery's capacity. So, if you have a 250mAh battery and the max charge current is specified as 1C, the charge current should be 250mA max, whereas with a 3000mAh battery, it should be 3000mA max.
If nothing is specified, assume 1C max current for both charging and discharging. If you have the time though, a lower max current will reduce the battery's aging.