How to Experiment with NumPy in Flux?

Introduction

Numpy is a python module widely used in scientific computing area and several customer have expressed they would like to use in their python script for Flux.

However, NumPy is a native C-Python module whereas PyFlux scripts are executed by Jython, a Java Python interpreter, which is not able to execute C Python modules thus preventing NumPy usage with Flux.

Nowadays with the experimental JyNI (Jython Native Interface) project it becomes possible to run native C-Python extensions with Jython 2.7.2. In particular it becomes possible to use Numpy modules with Jython 2.7.2

Since Flux 2018.0 release, Flux has been upgraded to use Jython 2.7.2 and it becomes possible to use the JyNI library to experiment with native C Python modules like NumPy.

Step-by-step

The steps below describe how to install Numpy for Flux on Windows. Numpy was successfully tested with Flux by applying the exact steps described below.

It may also work by choosing different options, for example when installing Python 2.7.2, but we did not test all the possible choices.

Uninstallation existing Python

This step may not be necessary but Numpy has not been tested with Flux on a machine where Python 2.7.2 was coexisting with another version of Python.

Install Python 2.7.2

1. Choose single-user installation: "Just for me"
2. Installation Path: C:\Python27
3. Keep default installation

Install the required Python module

1. Setup a Cmd.exe window to install the Python modules
   1. Open a Cmd.exe windows and enter the following commands
      1. mkdir C:\Numpy4Flux
      2. cd C:\Numpy4Flux
      3. set PATH=C:\Python27;%PATH%
   2. Keep the Cmd.exe window open to execute the next steps

2. Install SetupTools 44.1.1
   1. Download: setuptools-44.1.1.zip
   2. From the downloaded zip file extract setuptools-44.1.1 to your working directory
   3. In the Cmd.exe window enter the following commands:
      1. cd setuptools-44.1
      2. python.exe setup.py intall

3. Install PIP 20.3.4
   1. Download: pip-20.3.4.tar.gz
   2. From the downloaded zip file extract pip-20.3.4 to your working directory (C:\Numpy4Flux)
   3. In the Cmd.exe window enter the following commands:
      1. cd pip-20.3.4
      2. python.exe setup.py install

4. Install Numpy 1.13.3
   1. Download the Numpy package from https://pypi.org/project/numpy/1.13.3/
      1. For Windows 64-bits: numpy-1.13.3-cp27-none-win_amd64.wh
      2. Copy the downloaded file to your working directory
   2. In the Cmd.exe window enter the following commands:

      1. C:\Python27\Scripts\pip.exe install numpy-1.13.3-cp27-none-win_amd64.whl

   3. Numpy is now installed in: C:\Python27\Lib\site-packages
   4. Test that Numpy is correctly installed by opening the interactive Python interpreter in the Cmd.exe windows: python.exe
   5. Then try to import the numpy module: import numpy
   6. If no error message is displayed the module has been successfully installed for Python
   7. Exit the Python interpreter: exit()

Install JyNI 2.7 alpha 5 in Flux

1. In your web browser navigate to: https://github.com/Stewori/JyNI/releases/tag/v2.7-alpha.5
2. Download both:
   1. The cross-platform JAR file: JyNI.jar
   2. The platform-specific archive for Windows 64-bits: JyNI-2.7-alpha.5-bin-win-amd64.zip
3. Install the JyNI.jar into Flux
   1. Copy JyNI.jar to: %FLUX_INSTALL_DIR%\Third_party\jython-2.7.2\javalib\JyNI.jar
4. Install the JyNI.dll folder to Flux
   1. Create a userdll directory in Flux installation directory: %FLUX_INSTALL_DIR%\Third_party\jython-2.7.2\userdll
   2. From JyNI-2.7-alpha.5-bin-win-amd64.zip extract the JyNI.dll folder to: %FLUX_INSTALL_DIR%\Third_party\jython-2.7.2\userdll\JyNI.dll

Configure Flux custom directories

Configure the Flux Supervisor options to use your UserDLL and UserLib folders for all Flux modules (2D, 3D, Skew, PEEC):

1. Starts Flux Supervisor
2. Open the Options dialog by clicking on the Options button in the bottom left
3. Navigate to the Access Paths > Python options
4. For every Flux application (2D, 3D, Skew and PEEC) enther the following paths:
   1. User Python scripts directory : C:\Python27\Lib\site-packages
   2. User libraries directory : %FLUX_INSTALL_DIR%\Third_party\jython-2.7.2\userdll
   
   

   

   
   

5. Click Apply and close with OK

Test NumPy in Flux

1. From FluxSupervisor open Flux by clicking on Start a new project
2. In the PyFlux Command window enter the following Python command
   
   

   

   
   
3. You should obtain the following output in the Output console

   [[ 0 1 2 3 4]

   [ 5 6 7 8 9]

   [10 11 12 13 14]]

Troubleshooting

• Flux does not start any more using module Y (2D,3D,Skew,PEEC) after I configured NumPy for module X
  • What to do?

    Check you have configured the userdll directory for the module Y you want to start.

  • ExplanationOnce you have copied the JyNI.jar in %INSTALLFLUX%\Third_party\jython-2.7.2\javalib, Flux requires access to the JyNI.dll in order to start. This is the reason why, in the Flux Supervisor, the userdll directory must be configured for all modules (2D, 3D, Skew, PEEC) you want to use.

    It is understood this behaviour is not satisfying and should be improved in the future.

• Flux Server does not work any more after I configured NumPy
  • What to do?

    In order to fix the FluxServer you need to remove the JyNI.jar from the directory: %INSTALLFLUX%\Third_party\jython-2.7.1\javalib

    Obviously, doing this will "break" the usage of NumPy in Flux.

    This is a known limitation: it means that currently the user has to choose between using NumPy in Flux, or use the Flux Server but he cannot do both at the same time.

  • Explanation

    The reason is the same as the previous issue: once JyNI.jar is added to the directory it will be automatically loaded by Flux Server as well and Flux Server would need access to the JyNI.dll in order to start.

    However for Flux Server there is currently no way for the user to customize the directory where the FluxServer should look for the additional required DLL.