Recognizing feature points in the tool path is a crucial process in Computer Numerical Control (CNC) machining. Traditional methods rely on geometric descriptors and given thresholds to identify feature points, which cannot be automated due to the need for threshold selection. Recently, a new approach was introduced that uses deep learning to recognize feature points by converting Cutter Location (CL) points into images and using Convolutional Neural Networks. However, this method requires time-consuming preprocessing and large size storage of the model. To address this issue, we propose a novel lightweight deep learning-based method that efficiently recognizes feature points with significantly shorter preprocessing time. Our method encodes CL points as matrices and stores them as text files. We have developed a neural network with an Encoder-Decoder architecture, named EDFP-Net, which takes the encoding matrices as input, extracts deeper features using the Encoder, and recognizes feature points using the Decoder. Our experiments on industrial parts demonstrate the superior efficiency of our method.