A Novel Fixed-Parameter Activation Function for Neural Networks: Enhanced Accuracy and Convergence on MNIST | ||
| Journal of Innovations in Computer Science and Engineering (JICSE) | ||
| دوره 3، شماره 1، شهریور 2025، صفحه 53-58 اصل مقاله (491.59 K) | ||
| نوع مقاله: Original Article | ||
| شناسه دیجیتال (DOI): 10.48308/jicse.2025.241737.1086 | ||
| نویسندگان | ||
| Najmeh Hosseinipour-Mahani* 1؛ Amirreza Jahantab2 | ||
| 1Department of Applied Mathematics, Graduate University of Advanced Technology, Kerman, Iran | ||
| 2Department of Computer science, Shahid Bahonar University of Kerman Kerman, Iran | ||
| چکیده | ||
| Abstract— Activation functions are essential for extracting meaningful relationships from real-world data in deep learning models. The design of activation functions is critical, as they directly influence the performance of these models. Nonlinear activation functions are commonly preferred since linear functions can limit a model’s learning capacity. Nonlinear activation functions can either have fixed parameters, which are predefined before training, or adjustable ones that modify during training. Fixed-parameter activation functions require the user to set the parameter values prior to model training. However, finding suitable parameters can be time-consuming and may slow down the convergence of the model. In this study, a novel fixed-parameter activation function is proposed and its performance is evaluated using benchmark MNIST datasets, demonstrating improvements in both accuracy and convergence speed. Abstract— Activation functions are essential for extracting meaningful relationships from real-world data in deep learning models. The design of activation functions is critical, as they directly influence the performance of these models. Nonlinear activation functions are commonly preferred since linear functions can limit a model’s learning capacity. Nonlinear activation functions can either have fixed parameters, which are predefined before training, or adjustable ones that modify during training. Fixed-parameter activation functions require the user to set the parameter values prior to model training. However, finding suitable parameters can be time-consuming and may slow down the convergence of the model. In this study, a novel fixed-parameter activation function is proposed and its performance is evaluated using benchmark MNIST datasets, demonstrating improvements in both accuracy and convergence speed. | ||
| کلیدواژهها | ||
| Keywords—Activation Function؛ Deep Learning؛ Fixed-Parameter؛ Neural Networks؛ MNIST Dataset؛ Nonlinear Function؛ Gradient Optimization؛ Vanishing Gradient Problem | ||
|
آمار تعداد مشاهده مقاله: 189 تعداد دریافت فایل اصل مقاله: 202 |
||