Azure again

It looks like for using one ore more GPU's on Azure, one cannot just use a NVidea based docker with normal tensorflow-gpu: special  azure Python packages and code are needed to be able to access the GPU's.

First the azureml-sdk must be piped:

pip install -upgrade azureml-sdk
According to this blog these libs should be included in python

import azuremlfrom azureml.core import Experimentfrom azureml.core import Workspace, Runfrom azureml.core.compute import ComputeTarget, AmlComputefrom azureml.core.compute_target import ComputeTargetException

ws = Workspace.from_config()

exp = Experiment(workspace=ws, name='my experiment name')

Running on a GPU-enabled Azure Machine Learning compute cluster:

cluster_name = "gpucluster"
try:
    compute_target = ComputeTarget(workspace=ws, name=cluster_name)
    print('Found existing compute target')except ComputeTargetException:
    print('Creating a new compute target...')
    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_NC6',
                                                           max_nodes=4)

    compute_target = ComputeTarget.create(ws, cluster_name, compute_config)

    compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)


Tensorflow is retrieved from azureml: from azureml.train.dnn import TensorFlow
script_params = {
    '--data-folder': dataset.as_named_input('mnist').as_mount(),
    '--batch-size': 50,
    '--first-layer-neurons': 300,
    '--second-layer-neurons': 100,
    '--learning-rate': 0.001
}

est = TensorFlow(source_directory=script_folder,
                 entry_script='keras_mnist.py',
                 script_params=script_params,
                 compute_target=compute_target,
                 pip_packages=['keras', 'matplotlib'],
                 use_gpu=True)

run = exp.submit(est)
run.wait_for_completion(show_output=True)